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EXECUTIVE  SUMMARY 


New  human  estimates  for  chlorine  inhalation  lethality  as  a  function  of  exposure  duration  (for  a  healthy 
subpopulation  and  the  general  population)  were  derived  via  a  review  and  statistical  analysis  of  existing 
mammalian  lethality  data.  Median  lethal  dosage  and  quantal  response  data  were  found  and  analyzed  for 
eight  species  (mouse,  rat,  guinea  pig,  rabbit,  cat,  dog,  goat,  and  sheep)  and  for  durations  (T)  from  8  min  to 
235  min.  Resulting  human  estimates  were  expressed  via  the  toxic  load  model,  L(CnT)50  =  k,  where  n  = 
2.75  and  k  =  6.79  x  1010  (military)  or  2.58  x  1010  (general),  with  C  in  mg/m3  and  T  in  minutes.  For  2-min 
exposures,  LCT50  equals  13,500  and  9500  mg-min/m3,  for  military  and  general  population,  respectively. 
The  base  10  probit  slope  (concentration)  was  estimated  to  equal  8.0  (military)  and  6.0  (general).  Previous 
human  estimates  were  reviewed,  and  one  study  identified  as  corresponding  to  the  lower  confidence  limit 
for  the  new  estimate.  The  impact  of  the  new  estimate  was  evaluated  through  a  series  of  transport  and 
dispersion  modeling  runs  using  the  general  population  estimate.  The  predicted  downwind  hazard 
distances  were  consistent  with  what  has  been  witnessed  historically. 


REVIEW  AND  ASSESSMENT  OF  CHLORINE  MAMMALIAN  LETHALITY 
DATA  AND  THE  DEVELOPMENT  OF  A  HUMAN  ESTIMATE 

STATISTICAL  ANALYSIS  OF  CHLORINE  LETHALITY  DATA 


1.0  Introduction 

Chlorine  is  a  greenish-yellow,  highly  reactive 
halogen  gas  that  has  a  pungent,  suffocating  odor.  It 
is  widely  used  in  industry  and  as  a  biocide  in  water 
and  waste  treatment  plants.  Because  of  its 
extensive  use  and  toxicity,  it  has  been  the  subject 
of  many  toxicological  studies,  human  toxicity 
estimates  (ranging  from  threshold  mild  effects  to 
lethality  for  various  populations)  and  other  general 
reviews  and  commentary.  Many  of  these  previous 
efforts  have  focused  on  creating  toxicity  estimates 
that  would  be  appropriate  for  protecting  a  human 
population  from  an  accidental  chlorine  release. 
However,  many  risk  assessment  applications 
require  an  accurate  estimate  of  a  chemical’s  actual 
toxicity  and  thus  should  not  be  conservative  just  to 
be  on  the  safe  side.  At  present,  casualty  predictions 
for  accidental  chlorine  releases  are  at  odds  with 
what  has  been  observed  historically.  Either  the 
present  human  toxicity  estimates  are  too  low,  the 
currently  popular  atmospheric  transport  and 
dispersion  (ATD)  models  cannot  adequately  model 
chlorine  releases,  or  both. 


The  purpose  of  this  study  is  to  reevaluate  the 
human  lethality  estimate  as  a  function  of  duration 
for  chlorine  inhalation  exposures  involving  a 
healthy  human  population.  These  estimates  are 
meant  for  use  in  performing  risk  assessments 
involving  chlorine  airborne  releases.  To 
accomplish  this,  an  allometric  modeling  approach 
was  used  to  extrapolate  a  human  estimate  from 
suitable  data  in  the  existing  mammalian  toxicity 
database.  Included  in  the  analysis  is  the  estimate  of 
a  probit  slope  for  a  healthy  (military-type)  adult 
population.  In  addition,  estimates  for  the  general 
population  were  also  developed  based  upon  the 
military-type  estimates  using  the  mathematical 
method  of  Crosier  (2007). 1  General  population 
estimates  are  compared  to  those  of  previous 
researchers. 
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2.0  Statistical  Background 

The  development  of  human  toxicity  estimates  is 
often  a  multidisciplinary  endeavor.  In  this  section, 
several  important  topics  are  presented  that  aid  in 
understanding  the  development  of  a  human  toxicity 
estimate  for  chlorine. 

2.1  Allometric  Modeling 

The  dependence  of  a  biological  variable  X  on  body 
mass  M  is  typically  characterized  by  an  allometric 
scaling  law  of  the  form: 

X=X0Mb  (1) 

where  b  is  the  scaling  exponent  and  X0  a  constant 
that  is  characteristic  of  the  kind  of  organism 
involved.2  These  two  coefficients  can  be  estimated 
via  the  linear  regression  fit  of  X  versus  M  on  a  log- 
log  scale,  where  the  slope  and  y-intercept  of  the 
resulting  line  equal  b  and  log(X0),  respectively.  For 
biological  parameters  (such  as  blood  volume  and 
oxygen  diffusing  capacity)  that  scale  as  a  function 
of  the  volume  of  the  animal,  b  is  predicted  and 
generally  observed  (from  experimental  data)  to 
equal  one.2 

In  inhalation  toxicology,  the  dose  received  by  an 
animal  is  directly  related  to  the  amount  of  air 
inhaled.3'4'5  Minute  volume  ( Vm)“  is  a  commonly 


used  respiratory  measurement,  and  Bide  et  alb 
have  reviewed  the  existing  mammalian  minute 
volume  data  (as  well  as  species  body  mass)  for  the 
purpose  of  supporting  inhalation  toxicology 
research.  Table  2-1  lists  some  average  minute  vol¬ 
ume  and  species  body  mass  values  for  non- 
anesthetized  mammals.  An  empirical  relationship 
(for  both  anesthetized  and  nonanesthetized 
animals)  of  the  minute  volume  as  a  function  of  M 
has  also  been  developed.6  Based  upon  allometric 
scaling  law,  VM  should  scale  to  the  0.75  power,  but 
Bide  et  al.6  found  that  Vm  scales  to  the  0.809  power 
based  upon  the  available  experimental  data  for 
nonanesthetized,  adult  mammals: 

VM  =  (0.499) /W(0  809)  (2) 

where  VM  is  in  T./min  and  M  is  in  kg.  Bide  et  al. 
calculated  a  value  of  15.5  T./min  for  a  70  kg  man 
from  eq  2.  This  is  very  close  to  the  standard  value 
of  15  L/min  for  light  activity  used  by  DoD  as  a 
basis  for  their  toxicity  estimates  for  a  70  kg 
soldier.7  Bide  et  al.  further  note  that  their  equation 
is  for  normally  breathing,  nonanesthetized  mam¬ 
mals  engaged  in  mild  activity;  it  is  not  a  resting 
value. 


Table  2-1.  Summary  of  mammalian  (nonanesthetized)  minute  volume  values  from  Bide  et  al. 


(2000) 


SPECIES 

MASS 

(KILOGRAMS) 

MINUTE  VOLUME 
EXPT.  AVERAGE 

(LITERS) 

MINUTE  VOLUME 
ALLOMETRIC  MODEL  FIT 

(CUBIC  METERS) 

Vm  RATIO 
(EXPT./FIT) 

Mouse 

0.0207 

0.0310 

0.0217 

1.431 

Rat 

0.261 

0.203 

0.168 

1.206 

Guinea  Pig 

0.323 

0.182 

0.200 

0.910 

Rabbit 

2.77 

1.453 

1.138 

1.277 

Cat 

3.36 

0.845 

1.330 

0.635 

Dog 

13.0 

4.230 

3.975 

1.064 

Goat 

36.9 

9.600 

9.243 

1.039 

Sheep 

52.6 

13.90 

12.31 

1.129 

Human  (Male)* 

70 

15 

15.5 

0.967 

*  US  Army  Chemical  School,  Potential  Military  Chemical/Biological  Agents  and  Compounds ,  FM  3-1 1.9.  US  Army  Training  and  Doctrine 
Command,  Fort  Monroe,  VA,  10  January  2005. 


JMinute  volume  equals  the  tidal  volume  multiplied  by  the  number  of  breaths  per  minute.  The  tidal  volume  equals  the  volume  of 
air  inspired  or  expired  during  each  breath. 
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For  typical  whole  body  exposures,  the  exact 
dosage  inspired  by  an  individual  animal  is  usually 
not  measured.  In  place  of  the  exact  dosage,  a 
nominal  dose  (mg)  can  be  calculated  by 
multiplying  the  lethal  inhalation  dosage,  (mg- 
min/m3)  (estimated  from  experimental  toxicology 

data)  by  the  minute  volume  (m3/min): 

LD  50  =  VM  x  LCT50  (3) 

where  LD50  is  the  nominal  dose  lethal  to  50%  of 
exposed  individuals  and  LCT50  is  the  lethal 
concentration-time  (or  inhalation  dosage)  to  50% 
of  exposed  individuals.  If  the  LD50  is  expressed  on 
a  per  unit  mass  basis  (or  0.809  -  1),  then  it  will 
scale  to  the  -0.19  power  based  upon  eq  2  and  eq  3 
[and  assuming  that  the  observed  LCT50  values  (at 
some  set  duration)  do  not  vary  between  different 
species  for  a  particular  toxicant].  So,  in  such  a 
situation,  larger  mammals  are  more  sensitive  than 
smaller  mammals. 

However,  many  other  factors  influence  toxicity, 
and  some  of  these  factors  (i.e.,  metabolic, 
circulatory,  etc.)  may  also  scale  allometrically  in 
their  effect.  Allometric  modeling  can  be  used  to 
empirically  account  for  the  cumulative  effect  of 
these  factors  and  scale  the  nominal  LD50  (mg)  as  a 
function  of  species  body  mass,  as  shown  in  eq  4.  A 
human  estimate  can  then  be  obtained  from 
mammalian  data  by  extrapolating  to  a  70  kg 
human. 

LD50  =  X0  Mb 

or  (4) 

log(LD50)  =  log(X0)  +  b\og(M) 

In  eq  4,  both  the  regressor  term  (LD50)  and  the 
predictor  term  ( M)  are  both  highly  correlated  with 
the  parameter,  VM-  Controversy  exists  over 
regressions  involving  predictor  and  regressor 
variables  sharing  a  common  term,  which  dates 
back  to  1897s  and  has  continued  to  this  day.9'22  The 
common  charge  is  that  such  regressions  return 
spurious  correlations.  It  is  not  within  the  scope  of 
the  present  work  to  address  this  controversy  in 
detail.  The  statistical  community  has  rejected  the 
notion  that  these  types  of  regression  are 
statistically  invalid,  but  many  individuals  in  the 
greater  scientific  community  have  not  kept  abreast 
of  these  statistical  advances.9,10’13 
Nonetheless,  care  still  must  be  taken  with  these 
types  of  regressions.  According  to  Prairie  and 


Bird,13  the  correlation  between  such  composite 
variables  is  always  legitimate  provided:  (1)  they 
satisfy  the  assumptions  of  correlation  analysis;  (2) 
the  variables  are  meaningful,  that  is  they  represent 
the  concepts  of  interest  and  not  just  a  component 
of  them;  and  (3)  the  variables  do  not  share  a  large 
measurement  error  term.  In  the  present  work,  the 
first  point  is  addressed  in  the  data  analysis  and 
results  sections  of  this  report.  The  second  point  is 
satisfied — the  variables  do  properly  represent 
concepts  of  interest  for  this  problem.  For  the  last 
point,  the  measurement  error  is  vastly 
overshadowed  by  the  amount  of  random  error 
present  in  the  values  for  LCT50  (and  hence  the 
nominal  LD50),  M,  and  VM ■  Also,  though  VM  is 
correlated  with  M,  these  parameters  are  (in 
practice)  measured  separately  and  independently  of 
each  other — they  do  not  share  a  measurement  error 
term. 

As  a  further  precaution,  LC50  was  substituted  for 
LD50  in  eq  4.  Predictions  from  this  fit  were  then 
compared  to  the  LD50  fit  as  a  check  of  the 
appropriateness  of  the  LD50  model. 

2.2  Inhalation  Dose-Response  Statistics 
and  the  Toxic  Load 

For  each  individual,  there  is  a  dose  or  dosage  that 
is  just  sufficient  to  produce  a  specified  biological 
response.  These  just-sufficient  dosages  are  called 
effective  dosages  (ED)  to  distinguish  them  from 
administered  dosages.  The  statistical  properties  of 
this  response  distribution  have  been  extensively 
reviewed23,24  and  summarized.25,26  One  of  the  more 
important  aspects  of  effective  dosages  is  that  the 
distribution  of  effective  dosages  for  a 
homogeneous  population  is  usually  lognormal.  The 
severity  and  probability  of  effects  experienced 
depends  primarily  on  the  amount  of  toxic  material 
received.  When  the  just-sufficient  dosage  for  an 
effect  is  relatively  insensitive  to  exposure  duration, 
the  calculation  of  the  probability  of  effect  is 
straightforward.  However,  when  the  effective 
dosages  are  a  function  of  exposure  duration,  a 
different  approach  is  required.  Instead  of 
quantifying  the  amount  of  toxic  material  in  terms 
of  dosage,  a  new  term,  toxic  load  (TL),  has  been 
developed  and  extensively  used  in  inhalation 
toxicology  to  predict  the  probability  and  severity 
of  the  response.27'34  Toxic  load  is  normally 
expressed  as  some  function  of  vapor  concentration 
(C)  and  exposure  duration  (I),  with  toxic  load 
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equaling  CnT  being  a  typical  form  and  n  being  the 
toxic  load  exponent.  Sommerville  et  al ,33  provide  a 
summary  of  the  theory  and  application  of  the  toxic 
load  model. 

The  same  statistical  theory  used  for  measures  of 
dose  and  dosages23  has  also  been  adopted  for  use 
with  toxic  load.29'31'35'40  The  following  discussion 
has  been  largely  extracted  from  a  brief  review  by 
Crosier  and  Sommerville25  of  the  statistical  theory 
behind  dose  response.  Where  the  substitution  of 
toxic  load  for  dosage  is  not  straightforward,  it  is 
specifically  noted. 

A  plot  of  the  density  function  for  the  normal 
distribution  of  log(effective  dosage),  produces  the 
well-known  bell-shaped  curve.41  The  two  para¬ 
meters  most  often  used  to  characterize  a  normal 
distribution  are  its  mean  p  and  variance  a2  (or 
standard  deviation,  a).42  Using  these  two  para¬ 
meters,  every  normally  distributed  random  variable 
X can  be  standardized  to  a  standard  normal  random 
variable  Z: 

zJ-^xA  (5) 

a 

In  toxicology,  X  is  usually  the  common  logarithm 
of  the  dosage. 

Although  statisticians  typically  describe  the 
lognormal  distribution  of  effective  dosages  by  the 
mean  and  variance  of  log(ED),  toxicologists 
usually  describe  the  distribution  by  the  median 
effective  dosage,  ED50,  and  the  probit  (or  Bliss) 
slope,  nr. 

ED50  =  antilog(q)  (6) 

m  =  1/ct  (7) 

where  q  is  the  median  of  log(effective  dosage). 
The  median  effective  dosage,  ED50,  is  the  dosage  at 
which  50%  of  the  exposed  individuals  will  exhibit 
a  specified  biological  response.  For  vapor 
exposures,  the  equivalent  dosage  term  is  the  50% 
effective  concentration  (EC5o)  or  the  product  of  the 
exposure  concentration  and  exposure  time 
(ECT50).25  Median  effective  dosages  are  in  the 
original  units,  not  in  logarithms  of  the  original 
units,  and  hence  are  easier  to  interpret  than  p. 
Although  the  mean  p  and  median  q  of  a  normal 
distribution  are  the  same  (p  =  q),  this  property 
does  not  hold  for  a  lognormal  distribution. 


Effective  dosages  for  response  levels  other  than 
50%  can  be  calculated  from  p  and  a,  by  solving 
for  Win  eq  1  and  using  the  Z  value  corresponding 
to  the  cumulative  probability  of  interest.  The  50% 
response  level  corresponds  to  a  Z  value  of  zero. 
Tables  of  cumulative  probabilities  and  their 
corresponding  Z  values  are  found  in  standard 
statistical  textbooks41'42  or  obtained  using  statistical 
software.43 

Toxicologists  traditionally  use  base  10  logarithms 
to  calculate  the  probit  (Bliss)  slope,23'44  while 
engineers  often  use  natural  logarithms.  ’  ’  ’ 
Probit  slopes  based  on  both  natural  and  base 
1 0  logarithms  are  found  in  the  literature.  Care  must 
be  exercised  when  comparing  probit  slopes  from 
different  sources. 

The  probit  slope  equals  the  number  of  standard 
deviations  (A Z)  corresponding  to  a  factor  of  either 
10  or  e  (=  2.718  .  . )  change  in  ED.45  Thus,  a  probit 
slope  (base  10)  of  six  means  that  a  factor  of  10 
change  in  ED  corresponds  to  six  standard 
deviations  (A Z  =  6).  For  the  normal  distribution,  a 
range  of  Z  from  negative  four  (very  sensitive 
individuals)  to  four  (highly  tolerant  individuals)  (or 
A Z  =  8)  encompasses  over  99.99%  of  the  total 
population.  If  the  toxicant  has  a  probit  slope  of 
eight,  a  factor  of  10  separates  the  effective  dosages 
for  these  two  Z  values.  The  higher  the  probit  slope, 
the  closer  the  two  tails  of  the  distribution  are  in 
terms  of  ED  (in  other  words,  there  is  less  variance 
in  the  effective  dosages  of  the  population). 

Though  the  normal  distribution  is  continuous, 
quantal  data  (response  versus  no  response)  are 
used  to  estimate  the  parameters  (median  and  probit 
slope)  of  the  distribution  of  effective  dosages.23’46 
Probit  analysis  and  maximum  likelihood 
estimation  (MLE)  are  used  to  estimate  these 
parameters  from  experimental  data.23'29'47  The 
following  equation  is  fitted  via  probit 
analysis/MLE  for  vapor  toxicity  studies:23'29'30'31 

Yn  =  (Yp  -  5)  =  ka  +  kc  logC  +  k,  logT  (8) 

where  YN  is  a  normal  probability  unit  (or  normit), 
YP  is  a  probability  unit  (or  probit),  the  k' s  are  fitted 
coefficients,  C  is  vapor  concentration,  and  T  is 
exposure  time.  Normit  values  of  -2,  -1,  0,  1  and  2 
correspond  to  percent  effect  probabilities  of  2.3, 
15.9,  50.0,  84.1  and  97.7%,  respectively.  The 
constants  kc  and  k,  are  the  probit  slopes  for 


2-3 


concentration  and  time,  respectively.  The  toxic 
load  exponent  ( n )  equals  the  ratio  of  these  two 
slopes,  or  ( kjk, ).  Often,  experiments  are  conducted 
with  exposure  time  held  constant,  which  reduces 
eq  8  to  the  traditional  probit  equation.23  Thus,  the 
probit  slope  for  a  vapor  exposure  usually  refers  to 
the  slope  on  vapor  concentration  ( m  =  kc)  instead 
of  the  slope  on  exposure  duration.  Some  studies 
report  a  probit  slope  for  the  toxic  load.29'39'40'48'51 
Thus,  when  comparing  probit  slope  values  from 
various  studies,  a  common  basis  has  to  be  used. 
When  fitting  eq  8  or  similar  probit  model,  all 
variability  in  the  data  will  contribute  to  the 
estimate  for  m,  be  it  from  variance  due  to 
individual  susceptibilities,  batch  effects,  experi¬ 
mental  error,  differences  in  exposure  durations, 
etc.  Thus,  there  is  an  experimental  bias  towards 
producing  lower-than-actual  slope  estimates.  The 
precision  of  the  probit  slope  estimation  generally 
improves  as  more  subjects  are  used.  Probit  analysis 
performed  on  a  compilation  of  data  from  many 
sources  will  not  produce  an  accurate  measure  of 
variance  among  individuals  due  to  the 
heterogeneity  introduced  by  differences  among  the 
studies  (e.g.,  experimental  procedures,  type  of 
animals  used,  etc.).49  The  effect  of  such 
heterogeneity  will  be  to  lower  the  probit  slope. 
Also,  using  a  compilation  of  data  from  many 
sources  can  complicate  the  accurate  estimation  of 
the  toxic  load  exponent.29  For  studies  involving 
more  than  one  exposure  duration,  fitting  eq  8  with 
T  treated  as  a  factor  (instead  of  as  a  covariate)  will 
produce  the  best  overall  probit  slope  estimate  for 
the  whole  study.52,53 

A  straight  comparison  between  slope  values  can  be 
misleading  because  of  the  large  variability  among 
experimental  studies  (e.g.,  a  study  based  on  100 
animals  probably  has  a  more  precise  estimate  than 
a  study  based  on  30  animals).  In  the  present  work, 
a  different  approach  was  taken  towards  the  issue  of 
obtaining  an  average  probit  slope  value  from 
among  several  studies  (considering  all  the  potential 
pitfalls  mentioned  above).  For  each  study  where 
the  original  quantal  response  data  were  available,  a 
probit  slope  estimate  with  its  corresponding 
standard  error  was  generated  via  probit  analysis. 
Then,  a  weighted  average  of  all  the  individual 
probit  slope  estimates  was  taken  using  the  standard 
errors  of  the  individual  slope  values  as  the  basis  to 
generate  the  weights  (see  Section  5.1.1). 


When  toxic  load  is  being  used  as  the  measure  of 
toxicant  amount  received  by  an  individual,  eq  8  is 
usually  written  in  the  following  form: 

Yn  =  k0  +  kTL  log  TL  (9) 

with  kQ  and  kTL  being  fitted  coefficients,  and  kn_ 
being  the  probit  slope  on  a  toxic  load  basis. 

It  should  be  recognized  that  the  toxic  load 
relationship  (in  eq  9)  is  based  more  on  empirical 
observations  rather  than  on  basic  biological 
theories.29,30,31,35  As  a  result,  eq  9  needs  to  be 
derived  empirically  on  an  individual  toxicant  basis 
from  acute  toxicity  experiments  where  both  vapor 
concentration  and  exposure  duration  are  varied.54 
However,  the  value  for  the  toxic  load  exponent  can 
offer  some  insight  into  the  influence  of  underlying 
toxicological  mechanisms  (i.e.,  metabolic  detox¬ 
ification)  involved  in  a  particular  toxicant 
exposure.35  For  instance,  it  can  be  argued  that  if  the 
toxic  load  exponent  is  greater  than  one  for  a 
particular  chemical  exposure,  then  significant 
detoxification  may  be  occurring  during  the 
exposure,  since  in  this  situation  the  required 
effective  dosage  is  increasing  as  the  dosage  is 
incurred  over  a  longer  exposure  duration.  Slowing 
the  rate  of  dosage  administration  gives  any  existing 
detoxification  mechanism  a  chance  to  function, 
thereby  causing  an  increase  in  the  ECT50  as  a 
function  of  time. 

When  performing  an  allometric  regression  fit  of 
inhalation  dose  data,  eq  4  is  expanded  to  the 
following  form  to  account  for  the  time  dependency 
of  the  toxicity: 

log(LD50)  =  log(Y0)  +  km  log (M)  +  kt  log (7)  (10) 

Using  eq  10,  it  can  be  shown  algebraically  that  the 
toxic  load  exponent  (n)  equals  [1  /  (1  -  £,)].  Once 
the  appropriate  values  for  M  and  VM  are  substituted 
into  the  equation,  eq  10  can  be  written  in  a  toxic 
load  form  (eq  9). 

Fluctuations  in  the  concentration-time  profile  can 
have  a  major  impact  on  toxic  load  calculations. 
Unfortunately,  their  exact  impact  cannot  be 
accurately  determined  due  to  the  absence  of 
experimental  dose-population  response  curve  data 
taken  under  such  conditions  (see  Sommerville  et 
al ,33  for  a  review  of  the  issue).  The  theorized 
impact  for  chemicals  with  n  >  1  is  that  fluctuations 
will  increase  the  toxic  effect  of  the  inhalation 
exposure.  Fluctuations  are  usually  more 
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pronounced  in  the  area  around  the  point  of  release 
and  are  less  of  an  issue  further  downwind.  So,  the 
potential  impact  of  the  currently  perforce 
assumption  of  toxicity  being  independent  of 
concentration  fluctuations  will  likely  decrease  in 
magnitude  as  the  downwind  distance  increases. 

2.3  Statistical  Treatment  of  Historical  Data 

An  important  issue  when  analyzing  historical  data 
is  how  to  properly  judge  and  quantify  the  quality 
of  data  that  was  collected  over  a  span  of  decades, 
by  many  different  researchers,  and  among  a  wide 
variety  of  experimental  facilities.  It  is  difficult  to 
be  truly  objective  with  such  a  task.  With  linear 
regression  analysis,  the  use  of  weights  technically 
permits  putting  a  numerical  value  on  the  quality  of 
a  datum,  but  how  does  the  analyst  properly 
estimate  the  weight’s  value?  Often, a  measured 
response  does  have  an  estimated  error  associated 
with  it,  and  this  can  be  used  as  the  basis  for 
calculating  a  weight.  However,  this  type  of  error 
term  is  more  a  measure  of  precision  than  of 
accuracy,  and  who  is  to  say  which  laboratory  was 
more  accurate  50  years  ago  even  though  they  may 
have  been  less  precise?  A  solution  to  this  problem 
is  the  use  of  one  of  many  robust  regression 
methods  that  have  been  developed.47'55'56 

For  the  present  study,  the  iteratively  reweighted 
least  squares  (IRLS)47,55'56  method  was  used.  The 
IRLS  method  does  not  involve  making  an  a  priori 
decision  on  what  weight  values  to  use.  Instead, 
through  an  iterative  process,  the  weight  of  a  datum 
is  calculated  as  some  function  of  its  residual  (i.e., 
the  difference  between  the  actual  value  and  that 
predicted  from  the  regression  fit)  from  the  previous 
iterative  step.  Often,  the  standardized6  residual  (a 
residual  divided  by  an  estimate  of  its  standard 
deviation)  is  used,55  but  for  this  study  the 
studentized  deleted  residual  was  used  instead.  The 
studentized  deleted0  residual  of  an  observation  uses 
a  predicted  value  from  a  regression  fit  omitting  the 
observation,  and  it  is  more  sensitive  to  outliers 
than  the  standardized  residual.  Weights  based  on 
deleted  residuals  will  be  less  than  those  based  on 
standardized  residuals  (all  other  factors  being 
equal). 


2.4  Military  Subpopulation  versus  General 
Population  in  Toxicity  Sensitivity 

It  is  assumed  that  only  healthy  animals  were  used 
in  the  studies  reviewed  for  this  work. 
Consequently,  when  scaling  from  smaller 
mammals  up  to  humans,  the  derived  estimate 
should  correspond  to  a  healthy  human  sub¬ 
population.  In  this  case,  a  healthy  subpopulation  is 
being  defined  as  being  healthy  and  fit  enough  to 
serve  in  the  military  (which  would  be  healthier 
than  what  would  qualify  for  the  working 
subpopulation).  However,  this  does  not  address  the 
issue  of  the  more  vulnerable  portions  of  the  general 
population.  Several  studies  have  attempted  to 
account  for  this  difference  in  population 
sensitivity34'39'40’51’57'59  in  the  formulation  of  their 
toxicity  estimates.  For  this  study,  it  is  assumed  that 
the  vulnerable  subpopulation  can  be  adequately 
modeled  using  eq  5  if  a  probit  slope  and  median 
effective  dose  for  the  general  (or  whole) 
population  were  used.  The  vulnerable 
subpopulation  would  be  accounted  for  by  using  the 
lower  tail  of  the  bell  distribution.  Thus,  the  key  is 
to  change  the  basis  (from  healthy  subpopulation  to 
general  population)  of  the  probit  slope  and  median 
effective  dose  values  that  were  derived  using  the 
mammalian  data. 

In  a  series  of  three  technical  reports,1’26'60 
Edgewood  Chemical  Biological  Center  (ECBC) 
has  developed  a  mathematical  method  for  the 
conversion  from  one  population  basis  to  another, 
with  Crosier1  presenting  the  final  version  of  the 
method.  The  premise  of  the  approach  (see 
Appendix  I  for  a  more  detailed  description)  is  that 
mathematically  the  distribution  of  log(doses)  for  a 
healthy  subpopulation  is  located  completely  within 
the  distribution  formed  by  the  general  population 
(i.e.,  the  smaller  bell  curve  is  inside  the  larger  bell 
curve).  A  major  advantage  to  Crosier1  is  that  it 
reduces  the  amount  of  subjectivity  that  has  been 
used  in  the  past  in  accounting  for  population 
differences. 


bAlso  known  as  an  internally  studentized  residual. 

cAlso  known  as  a  deleted  t  or  externally  studentized  residual. 
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3.0  Historical  Chlorine  Data  and  Human 
Toxicity  Estimates 

3.1  Toxic  Action  of  Chlorine  Inhalation  on 
Animals  and  Humans 

Chlorine  is  an  acute  respiratory  irritant  in  both 
animals  and  humans,  producing  a  similar  spectrum 
of  toxic  responses  (from  minor  irritation  to  death) 
as  the  exposure  concentration  increases.27'34'58'61  It 
reaches  the  lungs  because  it  is  only  moderately 
soluble  in  water,  and  it  is  not  totally  absorbed  in 
the  upper  respiratory  tract  at  high  concentrations.58 
In  animal  studies,  the  cause  of  death  for  acute 
exposures  is  almost  always  recorded  as  pulmonary 
edema,  probably  due  to  damage  to  the  alveolar 
membranes.27’34  Occasionally,  reports  relating  to 
very  high  concentrations  of  chlorine  over  short 
exposure  durations  list  other  causes  of  death,  such 
as  broncho-constriction,  shock,  and  immediate 
respiratory  arrest.  Postmortem  examinations  of 
fatalities  resulting  from  accidental  or  wartime 
human  exposure  to  chlorine  have  revealed 
pulmonary  edema,  with  or  without  cardiac 
complications,  as  the  cause  of  death.  The  effect  of 
edema  is  to  cause  a  reduction  in  the  supply  of 
oxygen  from  the  lungs  to  the  blood.  Death  can 
occur  within  hours  or  may  not  occur  for  a  couple 
of  days  post-exposure.  Delayed  deaths  can  be  due 
to  secondary  pneumonia.  Thus,  effects  of  acute 
inhalation  are  qualitatively  similar  in  animals  and 
humans. 

It  has  been  concluded  by  the  Major  Hazards 
Assessment  Panel  (MHAP)  that  there  is  little  basis 
for  supposing  that  the  sensitivity  of  humans  to 
chlorine  is  significantly  different  from  that  of  small 
mammals.27'34  They  noted  several  factors: 

(1)  The  alveolar  membranes  between  humans  and 
mammals  are  not  likely  to  differ  significantly 
in  their  intrinsic  sensitivity  to  chlorine. 

(2)  The  volume  of  air  breathed  per  unit  time,  per 
unit  body  weight  for  mice  or  rats  at  rest  is 
approximately  ten  times  that  for  humans  at 
rest,  suggesting  that  small  animals  may  deliver 
a  higher  proportion  of  a  given  dose  of  chlorine 
to  their  lungs  and  thus  be  more  sensitive  than 
humans.  However,  active  humans  (in  an 
escape  scenario)  have  a  breathing  rate  that  will 
reduce  this  ratio  (when  compared  to  subdued 
rodents  in  experiments)  to  approximately  two 


or  less.  Thus,  differences  in  species  breathing 
rates  are  likely  to  have  little  impact. 

(3)  Rodents  are  obligate  nose  breathers,  whereas 
dogs  and  humans  are  not.  The  nose  has  the 
potential  for  removing  chlorine  before  it  can 
make  it  to  the  lungs.  However,  Alarie62  found 
that  the  LC50  for  tracheally  cannulated  mice 
was  about  two  times  lower  in  the  LC50  for 
normal  mice.  The  MHAP  concluded  that  this 
factor  difference  is  unlikely  to  lead  to  a 
significant  difference  between  rodents  and 
humans. 

3.2  Previous  Human  Lethality  Estimates 
for  Chlorine  Inhalation  Toxicity 

There  are  several  excellent  reviews  of  the  body  of 
chlorine  inhalation  toxicity  information  available. 
One  of  the  more  recent  is  Mannan,27  as  well  as  the 
1987  monograph  by  the  MHAP  sponsored  by  the 
Institution  of  Chemical  Engineers.34  Human 
estimates  from  five  previous  studies  were 
investigated  for  comparison  with  the  results  of  the 
present  work: 

(1)  Eisenberg  et  al51 

(2)  Perry  and  Articola63 

(3)  ten  Berge  and  van  Heemst51 

(4)  Rijnmond  Report/Harris  and  Moses59’64 

(5)  Withers  and  Lees39'40 

All  five  studies  recognized  that  the  inhalation 
toxicity  of  chlorine  varies  with  exposure  duration, 
and  they  used  the  toxic  load  model  to  describe  the 
relationship  (see  Section  2.2).  Nonetheless,  these 
studies  do  differ  in  their  philosophies  on  how  to 
best  extrapolate  from  animal  data  to  produce  a 
final  human  estimate. 

Eisenberg  et  al .57  and  Perry  and  Articola63  are 
related  works  that  were  written  in  support  of  the 
U.S.  Coast  Guard’s  Vulnerability  Model,  with 
chlorine  being  one  of  many  chemicals  for  which 
human  toxicity  estimates  were  collected  or 
derived.  They  reviewed  the  existing  mammalian 
toxicity  database  but  used  only  the  data  for 
estimating  the  variability  of  individual  toxic 
responses  and  disregarded  the  absolute  con¬ 
centration  values.34  Though  not  explicitly  stated, 
their  absolute  levels  were  ultimately  based  upon 
the  publication  by  Flury  and  Zemick.  34'65  Their 
human  estimates  lie  well  below  the  experimental 
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animal  data,  and  several  papers  do  not  recommend 
their  estimates  for  risk  assessment.34'39,40,51'59 

The  other  three  studies39,40,51,59,64  also  reviewed  the 
existing  mammalian  toxicity  database,  as  well. 
However,  unlike  Eisenberg  et  al.  and  Perry  and 
Articola,  they  used  the  database  for  estimating  both 
response  variability  and  absolute  levels  of 
concentration.  As  a  result,  more  realistic  estimates 
were  produced. 

ten  Berge  and  van  Heemst51  assumed  that  humans 
can  be  modeled  to  be  as  sensitive  as  the  average 
mammal,  which  meant  placing  the  human  estimate 
between  that  of  the  mouse  (more  sensitive)  and  the 
dog  (less  sensitive).  Their  estimate  of  the  human 
response  variability  was  for  a  general  population 
basis.  The  toxicity  time  dependence  was  based 
upon  mouse  data66  and  human  volunteer  irritation 
dose-response  relationship.67 
Harris  and  Moses59,0  took  the  work  of  Eisenberg  et 
al.  and  Perry  and  Articola  and  then  proceeded  to 
make  reasonable  adjustments  to  account  for  the 
absolute  levels  of  concentration  from  the 
mammalian  database  (which  was  previously 
neglected  by  Eisenberg  et  al.  and  Perry  and 
Articola).  However,  Harris  and  Moses  did  not 
make  any  explicit  statement  on  what  animal  model 
they  based  their  adjustments  on.  Since  their  human 
estimate  curve  is  very  similar  to  that  of  ten  Berge 
and  van  Heemst,  it  may  be  inferred  that  Harris  and 
Moses  assumed  that  human  toxicity  can  be  equated 
to  that  of  an  average  mammal.  As  for  human 
response  variability,  Harris  and  Moses  believed 
that  more  variability  exists  in  the  human  general 
population  response  (based  upon  the  historical 
record)  than  was  originally  estimated  by  the  El.S. 
Coast  Guard  studies.  However,  the  Harris  and 
Moses  estimate  for  response  variability  is  less  than 
that  of  ten  Berge  and  van  Heemst.  Thus,  though 
the  predicted  median  lethal  dosages  from  the  two 
estimates  are  practically  identical  (for  all  exposure 
durations),  predicted  dosages  for  percent  effect 
levels  other  than  50%  will  increasingly  differ  as 
the  two  extremes  (zero  and  100%  response)  are 
approached. 


Withers  and  Lees39,40  gave  greater  weight  to  the 
dog  chlorine  data  in  the  development  of  their 
human  estimate,  while  not  neglecting  to  interpret 
the  dog  data  in  light  of  the  other  animal 
experimental  data.  Also,  they  developed  human 
estimates  as  a  function  of  the  type  of  human 
population  (regular,  average  and  vulnerable)  and 
two  levels  of  physical  activity  (base  and  standard, 
corresponding  to  minute  volumes  of  6  L/min  and 
12  L/min,  respectively).  All  of  the  Withers  and 
Lees  estimates  are  lower  than  those  for  the  general 
population  estimates  from  ten  Berge  and  van 
Heemst51  and  Harris  and  Moses,59  with  the 
difference  increasing  as  the  exposure  duration 
increases. 

Though  some34,39  have  observed  that  the  LC5o 
varied  as  a  function  of  species  size,  there  are  no 
known  studies  that  have  actually  attempted 
allometric  modeling  of  the  existing  database. 


dHan'is  and  Moses  and  the  Rijmnond  Report  have  identical  human  estimates  and  are  treated  as  synonymous  by  Mannan.27 
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4.0  Historical  Record  of  Chlorine  Releases 

The  historical  record  of  chlorine  industrial 
accidents  has  been  discussed  in  several 
sources.27,34'59'68  The  most  notable  statistic  for  the 
reported  incidents  of  outdoor  releases  (with  release 
amounts  up  to  50  metric  tons)  is  that  fatalities 
(with  one  exception)  occur  within  about  400  m  of 
the  actual  point  of  release,  and  more  often  within 
250  m.27'34,51  The  largest  number  of  fatalities  for  an 
outdoor  chlorine  release  is  around  60  from  a 
ruptured  storage  tank  in  Romania  in  1939. 68 
Furthermore,  it  has  been  estimated  that  there  are 
(on  average)  0.3  to  0.8  fatalities  per  metric  ton  of 
an  accidental  industrial  release.34,59,68 

In  addition  to  the  above,  Marshall68  makes  the 
anecdotal  observation  that  no  civilian  casualties 
were  reported  as  a  result  of  the  first  large-scale 
chemical  warfare  attack  at  the  Second  Battle  of 
Ypres  on  22  April  1915  in  World  War  (WW)  I. 
The  Germans  released  168  tons  of  chlorine  over  a 
4-mi  front,  which  inflicted  an  estimated  5000 
military  fatalities.  The  distance  between  the 
German  and  Allied  lines  were  roughly  200  m  to 
600  m  apart  in  the  vicinity  of  the  attack.  The  Allied 
lines  were  located  about  4  mi  (or  6  km)  upwind 
from  the  town  of  Ypres,  which  was  still  occupied 
by  many  civilians.  It  might  be  inferred  from  the 
lack  of  evidence  that  the  chlorine  cloud  never 
reached  the  town,  since  it  is  likely  wartime 
propaganda  would  have  made  the  most  of  any 
civilian  deaths  had  they  occurred. 

Also  from  the  WWI  era,  a  chlorine  attack  was 
qualitatively  assessed  to  have  a  persistency  of 
approximately  1 0  min  in  the  open  and  3  h  in  woods 
under  calm  weather  conditions.69  High 
temperatures  and  wind  velocities  decrease 
persistency,  and  low  temperatures  and  wind 
velocities  increased  it.  Considering  the  size  of 
chlorine  releases  from  WW  I,  these  observations 
suggest  that  chlorine  releases  have  a  range  more 
limited  than  what  is  predicted  by  the  current  ATD 
models. 


In  developing  a  human  chlorine  toxicity  estimate, 
the  historical  record  must  be  considered.  It  is  well 
known  that  ATD  models  in  general  often  greatly 
overestimate  the  downwind  hazard  resulting  from 
chemical  releases.27  Many  factors  are  likely 
culprits  for  this  situation,0  with  the  toxicity 
estimates  being  high  on  the  list.  Any  chlorine 
toxicity  estimate  that  predicts  a  significant  number 
of  predicted  fatalities  located  further  than  a 
kilometer  downwind  from  the  release  point  should 
be  considered  suspect  based  upon  the  above 
historical  record.  A  significant  number  of  predicted 
fatalities  between  500  m  and  1000  m  should  be 
cause  for  concern.  A  case  in  point  is  that  of  Franks 
et  a/.49  Using  a  human  toxicity  estimate  from  the 
Health  and  Safety  Executive  (UK),61  Franks  et  al. 
produced  ATD  modeling  results  that  predicted  that 
the  downwind  range  of  a  LC95  would  be  from 
400  m  to  800  m  for  a  chlorine  puff  release  of 
1 000  kg  (a  small  amount  for  an  industrial-type 
accident),  and  for  the  LC50,  from  650  m  to  1 100  m 
(see  Section  7.5.4  for  more  details). 


eOther  factors  being  the  adequacy  of  the  algorithms  used  in  the  ATD  models  (for  dense  gas  releases,  terrain  effects,  etc.), 
possible  significant  atmospheric  decay  rates  of  the  released  chlorine  gas  due  to  its  reactivity,  and  incomplete  information 
about  the  actual  dispersion  of  chlorine  gas  clouds  from  the  accidental  releases. 
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5.0  Data  Acquisition 

An  extensive  literature  review  was  conducted  to 
locate  experimentally  measured  median  lethal 
dosage  values  for  mammalian  acute  inhalation 
exposures  to  chlorine.  Whenever  possible,  the 
underlying  raw  quantal  response  data  were  also 
obtained  if  available.  For  the  initial  identification 
of  relevant  studies,  three  reference  works  were 
consulted:  the  background  material  supporting  the 
Acute  Exposure  Guideline  Levels  (AEGL)  for 
chlorine;58  the  Third  Edition  of  Lee’s  Loss  Preven¬ 
tion  in  the  Process  Industries  (“Toxic  Releases,” 
Chapter  18);27  and  the  Chlorine  Toxicity  Mono¬ 
graph  published  by  the  MHAP.34  The  search  was 
subsequently  expanded  beyond  this  point. 

A  total  of  29  individual  median  lethal  inhalation 
dosage  (LCT5o)  values  were  collected  (or  derived 
from  raw  data)  from  18  studies  and  sources  for 
eight  mammalian  species  (see  Table  5-1)  for  a 
summary  of  the  collected  data).  Whenever 
possible,  original  sources  were  found  (and  cited) 
for  each  LCT50  value.  In  some  cases,  data  were 
discovered  in  previously  uncited  U.S.  Army 
technical  reports70'74  and  were  subsequently 
included  in  the  final  dataset.  In  addition  to  the 
collection  of  experimental  data,  previously  derived 
human  toxicity  estimates  for  chlorine  were  also 
reviewed  and  collected.39'40,51,57,63’64 

Some  of  the  LCT50  values  in  Table  5-1  do  not 
exactly  correspond  to  what  was  originally  reported. 
Many  older  studies  used  the  Litchfield  and 
Wilcoxon  method75  for  calculating  median 
effective  dosages  and  probit  slopes  from  binary 
response  data.  The  method  is  graphical  in  nature 
and  was  originally  developed  as  a  means  for  rapid 
calculations  in  the  days  before  widespread 
computer  usage.  Probit  analysis23  with  modem 
computer  software  is  more  accurate  and  quicker 
than  the  Litchfield  and  Wilcoxon  method.  Where 
the  original  response  data  were  available,  the 
originally  reported  LCT50  value  was  updated  via 
probit  analysis. 

Besides  the  collected  LCT50  values,  average 
species  body  mass  and  minute  volume  values  from 
Bide  et  al.6  for  nonanesthetized  mammals  were 
used  in  the  present  analysis  (see  Table  2-1).  For  a 
healthy  human,  an  average  body  mass  of  70  kg  and 
minute  volume  of  15  L/min  (for  light  activity) 
were  assumed. 


For  this  study,  vapor  concentration  values  are 
expressed  in  units  of  mg/m3.  In  a  majority  of  the 
cited  studies,  concentrations  are  expressed  in  parts 
per  million  (ppm).  The  following  was  used  to 
convert  from  ppm  to  mg/m3  for  chlorine: 
1.0  mg/m3  =  1  ppm  x  (2.898). 

Only  studies  where  mammals  were  exposed  for  a 
set  period  of  time  and  then  removed  from  the 
chamber  for  observation  were  used  in  the 
subsequent  statistical  analysis  (Section  7).  Studies 
that  exposed  the  animals  continuously  until  death 
was  observed  were  not  used.  Such  studies 
generally  produced  higher  LCT50  values  relative  to 
fixed  duration  and  observe  studies.  An  example  of 
an  exposure  to  death  study  is  Weedon  et  al.,ss 
which  is  cited  by  many  reviews  of  chlorine 
toxicity27,34,58  and  was  not  included  in  the  dataset 
for  this  study. 

In  addition,  many  sources  give  incomplete 
information  about  what  their  “lethal”  values  truly 
signify  (i.e.,  is  it  1%  lethality  or  50%  lethality?). 
Only  LCT50  values  were  included  in  the  allometric 
modeling  of  the  present  study.  If  an  LCT50  value 
could  not  be  estimated  from  any  reported  quantal 
response  data,  that  study’s  “lethal”  value  was  not 
included  in  the  present  dataset.  Also,  due  to  the 
recognized  importance  of  time  on  chlorine  toxicity, 
no  LCT50  value  was  included  unless  a  duration 
value  was  associated  with  it. 

Significant  statistical  advances  have  occurred  since 
most  of  the  reviewed  reports  were  published.  Many 
of  the  reports  were  published  on  or  before  the 
WWII  era.  Some  even  resulted  from  observed 
WWI  era  data  but  were  primary  qualitative  in 
nature.  The  approaches  used  by  the  authors  of 
these  reports  were  not  discemable  through  the  set 
of  documents  gathered,  and  the  methods  used  may 
not  have  been  nearly  as  sophisticated  as  those  that 
are  now  in  place  (e.g.,  an  LCT50  may  have  been 
simply  reported  for  a  dosage  that  marked  the 
demise  of  one  half  of  the  subjects  in  an 
experiment). 
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Table  5-1.  Data  and  calculations 


STUDY  (AUTHORS) 

REPORT 

DATE 

REF 

NUMBER 

RAW 

DATA 

METHOD 

PROBIT 

SLOPE 

SLOPE 

ERROR 

TIME 

(min) 

LCTso 

(mg- 

min/m3) 

LDso 

(mg) 

Mouse 

Silver,  McGrath 

1942 

76 

Yes 

Probit 

7.3207 

0.5184 

10 

13150 

0.41 

Silver,  McGrath,  Ferguson 

1942 

77 

Yes 

Probit 

7.8260 

1.0520 

10 

19430 

0.60 

Zwart,  Woutersen 

1988 

78 

Yes 

Probit 

10.0410 

2.0060 

10 

30620 

0.95 

Alarie 

1980 

62 

No 

10 

8752 

0.27 

Bunting 

1945 

79 

No 

10 

15200 

0.47 

Lipton,  Rotariu 

1941 

80 

No 

10 

18199 

0.56 

Bitron,  Aharonson 

1978 

66 

Yes 

MLE 

10.6300 

1.0160 

11 

9245 

0.29 

Schlagbauer,  Henschler* 

1967 

81 

Yes 

Probit 

6.1790 

1.2280 

30 

10682 

0.33 

Zwart,  Woutersen 

1988 

78 

Yes 

Probit 

6.8240 

3.6750 

30 

49300 

1.53 

Bitron,  Aharonson 

1978 

66 

Yes 

MLE 

10.6300 

1.0160 

55 

27096 

0.84 

Back,  Thomas,  MacEwen8 

1972 

82,83 

No 

60 

23822 

0.74 

Rat 

Zwart,  Woutersen 

1988 

78 

Yes 

Probit 

15.8410 

3.3290 

5 

77850 

15.80 

Zwart,  Woutersen 

1988 

78 

Yes 

Probit 

15.8410 

3.3290 

10 

61160 

12.42 

Zwart,  Woutersen 

1988 

78 

Yes 

Probit 

15.8410 

3.3290 

30 

54810 

11.13 

Zwart,  Woutersen 

1988 

78 

Yes 

Probit 

15.8410 

3.3290 

60 

83820 

17.02 

Vernot,  MacEwen,  HaunB 

1977 

82,83 

No 

60 

50947 

10.34 

Guinea  Pig 

Lehmann 

1882 

84 

No 

MLE 

30 

290000 

52.78 

Rabbit 

Marshall 

1917(a) 

70 

Yes 

MLE 

30 

21580 

31.36 

Barrow,  Smith 

1975 

85 

Yes 

MLE 

30 

30479 

44.29 

Lehmann 

1882 

84 

Yes 

MLE 

190 

300000 

435.90 

Cat 

Lehmann 

1882 

84 

Yes 

MLE 

235 

390000 

329.55 

Dog 

Armstrong 

1923 

71 

Yes 

Probit 

7.5120 

2.8290 

10 

58340 

246.78 

Marshall 

1917(a) 

70 

Yes 

Probit 

3.1170 

1.4460 

30 

47010 

198.85 

Underhill 

1920 

86 

Yes 

Probit 

3.4514 

0.8053 

30 

49380 

208.88 

Marshall 

1917(b) 

72 

No 

30 

75000 

317.25 

Beebe 

1924 

73 

No 

30 

90000 

380.70 

Sheep 

Batchinsky,  Martini, 
Jordan 

2006 

87 

No 

30 

24343 

338.37 

Goat 

McElroy,  Shils,  Ginsburg 

1943 

74 

Yes 

MLE 

18.5 

51500 

494.40 

McElroy,  Shils,  Ginsburg 

1943 

74 

Yes 

MLE 

8.5 

16500 

158.40 

f 

Schlagbauer  and  Henschler  also  collected  data  for  3-h  and  6-h  exposures,  but  their  quantal  response  data  did  not  bracket  the  50% 
level,  which  was  unsuitable  for  calculating  LCT50  values  at  these  durations. 
gBack  et  al.  and  Vemot  et  al.  report  the  same  data  for  both  mouse  and  rat. 
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6.0  Data  Reduction 


The  collected  data  were  subjected  to  several 
calculation  procedures  prior  to  use  for  the  final 
model  fit.  First,  if  sufficient  quantal  response  data 
were  available  from  an  individual  study,  both  the 
probit  slope  (with  respect  to  concentration)  and  the 
LCT50  were  estimated  via  probit  analysis.23  The 
calculation  was  performed  using  the  binary  logistic 
regression  routine  in  Minitab R  Statistical  Software, 
and  the  standard  error  of  the  probit  slope  recorded 
(see  Appendix  A  for  an  example  probit  analysis). 
If  more  than  one  exposure  duration  was  used 
within  a  study,  the  duration  was  treated  as  a  factor 
(not  a  covariate)  in  the  calculations  (see  Section 
2.2),  thereby  producing  a  best  estimate  for  the  true 
probit  slope  for  the  whole  study  (which  was  the 
case  with  Zwart  and  Woutersen).78  However,  in 
many  cases,  only  LCT50  values  were  explicitly 
given  in  the  publications  reviewed. 

For  four  studies, 70,74,84,85  only  limited  quantal 
response  data  were  recorded  for  the  most  part, 
which  was  not  sufficient  for  use  in  a  traditional 
probit  analysis.  No  probit  slope  or  LCT50  values 
were  originally  reported.  For  these  datasets,  a  one- 
factor  MLE  analysis47,53  with  a  probit  link  function 
was  used  to  estimate  the  LCT50.  An  example  of  the 
one-factor  MLE  analysis  is  shown  in  Appendix  B. 
The  MLE  calculations  for  Lehmann84  and  McElroy 
et  al .74  are  shown  in  Appendix  C. 

Once  the  LCT50  values  had  been  collected  or 
estimated,  they  were  converted  into  nominal  lethal 
doses  using  eq  4  and  the  species  appropriate 
average  experimental  minute  volume  values  from 
Bide,  et  al6  Table  5-1  displays  the  name  and  year 
of  the  study,  reference  number,  species, 
availability  of  raw  data,  the  method  of  LCT50 
estimation  (if  appropriate),  exposure  duration,  the 
value  of  the  collected  or  estimated  LCT50  value, 
and  estimated  probit  slope  value  with  associated 
standard  error. 

A  more  complete  table  that  provides  all  data — both 
observed  and  computed — is  given  in  Appendix  D. 
All  the  values  shown  in  Table  5-1  were  used  in  the 
final  model  fit  of  eq  10.  The  mammalian  LCT50 
values  are  shown  in  Figure  6-1,  as  well  as  the 
previous  general  population  estimates  listed  in 
Section  3.2. 
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a.  Median  lethal  dosage  (LCT50)  versus  exposure  duration  (7) 


Time  (minutes) 

b.  Median  lethal  concentration  (LC50)  versus  exposure  duration  (7) 

Figure  6-1.  Mammalian  experimental  data  and  previous  human  estimates  for  chlorine 
inhalation  lethality  as  a  function  of  exposure  duration11 


hThe  plots  for  the  Rijnmond  Report  and  ten  Berge  and  van  Heemst  are  essentially  identical,  though  ten  Berge  and  van  Heemst 
is  very  slightly  higher. 


6-2 


7.0  Data  Analysis  And  Results 

7.1  Probit  Slope  Estimation 

7.1.1  Slope  Estimation  via  a  Weighted 
Average 

As  explained  in  Section  2.2,  the  derivation  of  an 
appropriate  human  probit  slope  from  mammalian 
data  is  not  a  trivial  matter.  A  weighted  average  was 
taken  of  the  experimental  probit  slopes  listed  in 
Table  5—1.  The  inverse  of  the  variances  (the 
squares  of  the  standard  errors)  of  the  probit  slope 
estimates  were  used  for  the  weights.  Thus,  smaller 
weights  were  objectively  assigned  to  less  precise 
slope  estimates.  The  probit  slopes  from  two  WWI 
era  studies  (Marshall70  and  Underhill86)  were  not 
used  in  this  analysis.  As  mentioned  in  Section  2.2, 
other  factors  besides  variability  in  individual 
sensitivities  contribute  to  the  probit  slope  value 
and  can  “artificially”  suppress  the  slope  value.  It 
was  judged  that  such  was  the  case  with  the  slopes 
from  these  two  studies. 

The  weighted  average  of  the  probit  slope  values 
equals:89 


variances)  equals:89 


(12) 


where  cr|  is  the  variance  of  the  weighted  average. 

Table  7-1  presents  the  step-by-step  process  in 
developing  both  the  point  estimate  of  the  slope  and 
the  variance.  Below  the  chart  are  the  supplemental 
calculations  for  the  95%  confidence  interval  (based 
upon<r|)  for  the  probit  slope. 

The  computations  are  as  follows:  Weighted 
average  for  probit  slope  becomes  (using  eq  1 1): 

o  _  (54.07633)  _ 

V"  w  (6.794295) 

The  weighted  variance  is  simply  the  inverse  of  the 
sum  in  the  next  to  last  column  (see  eq  12),  which 
equals  0.147182.  Thus,  the  95%  confidence 
interval  for  the  weighted  slope  average  becomes: 


S  = 


Y"  w  s. 

1  ’ 


(11) 


(7.96)  -  (1.96)^(0.147182)  <  S  <  (7.96)  +  (1.96)^/(0.147182) 

or 

7.21  <S  <  8.71 


where  S  is  the  weighted  average  probit  slope,  5, 
are  the  individual  probit  slope  estimates,  and  w,  are 
the  weights.  The  variance  of  the  weighted  average 
(when  the  weights  are  based  on  the  individual 


It  is  assumed  that  the  above  weighted  average  for 
the  probit  slope  is  a  reasonable  estimate  for  the 
probit  slope  for  a  healthy  (i.e.,  military)  human 
subpopulation. 


Table  7-1.  Probit  slope  computational  scheme 


REF# 

COUNT 

PROBIT  SLOPE 
ESTIMATE 

STANDARD 

ERROR 

VARIANCE 

(  =  SF) 

WEIGHT 

(  =  i/V) 

PRODUCT 

(  =  W|  *  Si ) 

(Si) 

(SE) 

(V) 

(Wi) 

76 

1 

7.3207 

0.5184 

0.268739 

3.721089 

27.240974 

77 

2 

7.8260 

1.0520 

1.106704 

0.903584 

7.071448 

78 

3 

10.0410 

2.0060 

4.024036 

0.248507 

2.495256 

66 

4 

10.6300 

1.0160 

1.032256 

0.968752 

10.297833 

81 

5 

6.1790 

1.2280 

1.507984 

0.663137 

4.097524 

78 

6 

6.8240 

3.6750 

13.505625 

0.074043 

0.505271 

78 

7 

15.8410 

3.3290 

11.082241 

0.090234 

1.429404 

71 

8 

7.5120 

2.8290 

8.003241 

0.124949 

0.938620 

Column  Sums 

40.53083 

6.794295 

54.07633 

7-1 


7.1.2  Slope  Estimation  From  Other 
Sources 

Several  probit  equations  for  human  lethality  have 
been  developed  or  estimated  within  the  past 
30  years.  Five  such  relations  are  listed  in 
Table  7-2. 27,39,40’5 1  ■57-59'ft3’(’4  All  five  were  developed 
for  use  in  modeling  chlorine  toxicity  within  the 
general  population,  though  in  the  case  of  Withers 
and  Lees,4u  no  probit  relation  was  originally 
reported  for  the  general  (or  their  average) 
population.  Instead,  they  reported  estimated 
concentrations  for  10%,  50%,  and  90%  lethality 
for  two  separate  exposure  durations.  A  fit  was  then 
made  to  these  points  to  derive  their  equation  in 
Table  7-2.  Also,  Withers  and  Lees  developed 
probit  equations  for  the  healthy  and  vulnerable 
subpopulations. 

Four  of  the  five  equations  were  originally  reported 
using  probit  units,  natural  logarithms  and 
concentrations  expressed  in  ppm.  For  ECBC  risk 
assessment  and  toxicological  applications,  normit 
units,  base  10  logarithms  and  concentrations 
expressed  in  mg/m3  are  normally  used.  Also,  the 
probit  slope  based  on  concentration  ( kc )  is 
commonly  used  by  ECBC  and  others  rather  than  a 
slope  based  on  toxic  load  (kn).  So,  these  equations 
were  transformed  from  the  published  form  to  the 
construct  used  for  this  analysis  as  shown  below. 
The  resulting  relations  are  also  listed  in  Table  7-2. 

In(x)  =  ln(10)  x  log(x),  (13) 

C (ppm)  =  0.34507  x  C (mg/m3),  .... 

and  '  ' 

YP  (Probit)  =  Yn  (Normit)  +  5.  (15) 


Given  an  original  equation  in  the  form: 

Yp=  ka  +  kTL  In  (C"7),  (16) 

where  k0  is  the  constant,  kTi_  is  the  probit  slope 
(relative  to  the  toxic  load),  and  n  is  the  toxic  load 
coefficient,  the  construct  of  the  transformed 
equation  becomes: 

Yn  +  5  =  ka  +  kTL  {(ln(10)  x  log[(0. 34507  Cmg/m3)n  7] } 

Yn  +  5  =  k0  +  kTL  ln(10)  {log  [(0.34507  Cmg/m3)n]  +  log[7]} 

Yn  +  5  =  k0  +  kTL  ln(10)  {n  log[(0.34507  Cmg/m3)]  +  log[7] } 

Yn  +  5  =  k0  +  kTL  ln(10)  {n  log[0.34507]  +  n  log[Cmg/m3)]  + 
log[7]} 

Yn+  5  =k0  +  n  kTL  ln(10)  log  (0.34507)  +  kTL  ln(10)  [n 
l°g(C)  +  log(7)] 

where  the  concentration  C  is  now  expressed  in  mg 
per  m3.  Continuing, 

YN+5=k0  +  nkTL  ln(10)  log  (0.34507)  +  kTL  ln(10) 
[log(C")  +  log(7)] 

Yn  +  5  =  ka  +  n  kTL  ln(  10)  log(0. 34507)  +  kTL  ln(10) 
(log(CT)| 

Yn=  [ka- 5  +  n  kTL  (ln(10))  log(0.34507)  ]  +  \kTL  (ln(10))] 
log(C"7) 

or  in  simplified  notation: 


Yn  =  kQ'  +  kTL'  log(Cn7)  (17) 


Table  7-2.  Human  lethality  equations  from  other  sources 


REFERENCE 

REF# 

ORIGINAL  PUBLISHED  EQUATION* 

CONVERTED  EQUATION** 

Withers,  Lees 

39,40 

Yn  (Normit)  =  -1 1 .697  +  1 .675  log(C  2  00  7) 

Eisenberg,  Lynch, 
Breeding 

57 

Yp  (Probit)  =  -17.10  +  1.69  ln(C2-75  7) 

Yn  (Normit)  =  -27.045  +  3.891  log(C 2  75  7) 

Perry,  Articola 

63 

Yp  (Probit)  =  -36.45  +  3.13  ln(C2-64  7) 

Yn  (Normit)  =  -50.242  +  7.207  log(C 2  64  7) 

ten  Berge,  van  Heemst 

51 

Yp  (Probit)  =  -5.04  +  0.50  ln(C275  7) 

Yn  (Normit)  =  -1 1 .503  +  1 . 1 51  log(C  ^  7) 

Rijnmond  Report 
(Harris,  Moses) 

59,64 

Yp  (Probit)  =  -11.40  +  0.82  ln(C2-75  7) 

Yn  (Normit)  =  -18.799  +  1.888  log(C2  75  T) 

*The  concentration  "C"  is  measured  in 
parts  per  million 

**The  concentration  "C"  is  measured  in  milligrams 
per  cubic  meter 
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Reformatting  the  converted  equations  in 
Table  7-2: 

YN  =  k0'  +  kTL'  log(C'T) 

=  ko  +  kTL  log(C")  +  kTL  log(7) 


the  studentized  deleted  residuals  being  calculated 
for  the  set  of  observations.  From  these  residuals,  a 
set  of  weights  were  created  for  the  next  iteration. 
Iterations  continued  until  a  specific  tolerance  level 
was  reached. 

Weights  (W)  were  calculated  as  follows: 


=  ka'  +  nkTi  log(C)+  kTi  log(T) 

=  K  +  nk-n'  log(C)+  kTL  log (7)  +  nkTL'  log (7)  - 
nkTL  log(T) 


convergence 

criteria 


U  [1  +  TRES^] 
for  each  j  :  (l VU1J  -  W,  T  <  1x10~6 


(19) 


=  k„  +  nkTi  log(C)  +  nkTL'  log(7)  +  kTL'  log(7)  - 
n  kTL’  log  (I) 

=  k0}  +  nkTi  log  ( CT)  +  UT  (1  -  n)  log(7) 

The  important  element  of  this  configuration  is 
nkTL,  with  equals  the  probit  slope  (base  10)  relative 
to  concentration  (kc),  differentiated  from  the 
previously  discussed  probit  slope  relative  to  toxic 
load  (ku).  It  also  turns  out  that  the  probit  slope  for 
time  (kt — see  eq  8)  equals  kTL- 

Table  7-3  presents  the  probit  slopes  ( kc )  for  the 
references  given  in  Table  7-2. 

7.2  Linear  Regression  Analysis 

7.2.1  Fit  of  Mammalian  LD50  Values 

Using  eq  10  (see  Section  2.2)  and  IRLS  (see 
Section  2.3)  with  Minitab®,  the  following  model  fit 
was  obtained: 

log(LD50)  =  (0.515)  +  (0.869)log(M)  +  (0.630)log(r)  (18) 

The  details  of  the  model  fit  are  described  in 
Appendix  F.  A  plot  of  eq  18  (with  associated  95% 
confidence  limits)  is  shown  in  Figure  7-1  with  the 
individual  experimental  mammalian  LD50  values 
for  comparison. 

For  the  IRLS  process55  an  initial  unweighted  model 
fit  to  eq  10  was  performed  using  Minitab®,  with 

Table  7-3.  Probit  slopes  (base  10)  on  concentra¬ 
tion  from  other  sources  for  the  general 


population 


REFERENCE 

PROBIT  SLOPE 

(WITH  RESPECT  TO  CONCENTRATION) 

Withers,  Lees39'40 

3.3 

Eisenberg  et  at.57 

10.7 

Perry,  Articola63 

19.0 

ten  Berge,  van  Heemst51 

3.2 

Rijnmond59'64 

5.2 

where  the  counter  represents  ith  iteration,  and 
commences  at  2  since  the  first  regression  run  is 
unweighted,  “j”  is  the  index  for  the  /h  observation, 
and  TRES! ;  is  the  studentized  deleted  residual  for 

th  ,J  .  .  .  xh 

the  j  observation  within  the  /  iteration. 
Convergence  was  considered  reached  when,  for 
each  Wj  successive  iteration  (/  and  i  +  1),  estimates 
were  produced  that  agreed  to  the  sixth  decimal 
place 

Appendix  E  presents  the  procedure  within  a  flow 
diagram  context,  detailing  the  IRLS  process  used 
in  this  effort.  Appendix  F  gives  an  extract  of  the 
IRLS  session  that  culminated  in  the  regression 
equation  result  for  this  analysis. 

7.2.2  Fit  of  Mammalian  LC50  Values 

Other  models  can  be  used  to  fit  the  data,  and  two 
such  expressions  were  also  investigated  based 
upon  using  eq  10  (see  Section  2.2)  to  solve  for 
LC50  (instead  of  LD50).  For  the  first  fit,  log(LCso) 
was  regressed  against  log(AT)  and  log(7),  with  no 
attempt  to  model  the  nominal  respiratory  dose. 

In  the  second  fit,  log(LC50)  was  regressed  against 
log (M),  log(T),  and  log(Era„0).  The  minute  volume 
ratio,  Vmtio,  equals  the  experimental  average  of  VM 
values  for  a  species  (see  Table  2-1)  divided  by  VM 
from  the  allometric  model  fit  of  Bide  et  al 6  (see 
eq  2).  Conceptually,  Vrann  empirically  represents  a 
shape  factor  of  sorts — how  much  a  species 
respiratory  geometry  differs  from  the  norm. 
Theoretically,  Vmtio  should  not  be  correlated  with 
M  as  the  number  of  species  represented  in  a 
particular  dataset  increases.  However,  for  the 
species  represented  in  the  present  dataset,  there  is  a 
correlation  between  log (M)  and  log (Vratio).  Thus,  in 
the  final  model  fit  with  both  log (Af)  and  log(Vmtio), 
log(M)  was  found  to  be  statistically  insignificant, 
and  it  was  dropped  from  the  final  fit. 
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Figure  7-1.  Allometric  model  fit  of  nominal  inhalation  LD50  for  healthy  adult 
humans  as  a  function  of  exposure  duration 


Using  IRLS  (see  Section  2.3),  the  following  model 
fits  were  obtained  (with  weights  calculated  using 
eq  19): 

log^Cj,,)  =  (3.62)  +  ( 0.0976) I og(/W)  +  (-0.278)  log(7)  (20) 

log^)  =(431)  +(-103)109(14*,)  +  (-0.627) log(T)  (21) 

More  details  of  the  above  model  fits  are  listed  in 
Appendix  G.  A  plot  of  eq  20  and  eq  21  (with 
associated  95%  confidence  limits)  are  shown  in 
Figure  7-2  along  with  the  LD50  fit  (eq  19)  and  the 
individual  experimental  mammalian  LC50  values 
for  comparison.  The  95%  confidence  intervals  for 
eq  20  and  eq  21  are  roughly  equivalent  to  those  for 
eq  19,  with  the  ranking  of  relative  width  (from 
narrowest  to  widest)  being  eq  21  <  eq  19  <  eq  20. 
eq  19  was  converted  from  an  LD50  to  LCT50  basis 
(see  Section  7.3)  for  its  plot  in  Figure  7-2. 

7.2.3  Summary  of  Linear  Regression  Fits 

Chlorine  toxicity  does  correlate  (with  statistical 
significance,  p-value  <  0.001)  with  species  body 
mass.  On  an  absolute  nominal  dose  basis,  LD50 
scales  to  A/0  87.  Thus,  lethality  for  larger  animals 
requires  a  greater  agent  dose  on  an  absolute  basis. 
Flowever,  on  a  per  kg  of  body  mass  basis,  larger 
mammals  are  more  sensitive  to  chlorine  than 
smaller  mammals  [LD50  (mg/kg)  scales  to  AT0 13]. 


The  LC5o  scales  to  AT0  10  (with  a  p-value  =  0.01 1) — 
the  LC50  is  not  a  very  strong  function  of  species 
body  mass.  In  practical  terms,  this  relationship 
predicts  that  the  healthy  human  LC50  value  is  a 
factor  of  2.25  higher  than  the  mouse  and  just 
barely  greater  than  the  LC5o  value  of  the  dog  (a 
factor  of  1.18)  or  the  goat  (a  factor  of  1.07). 

The  regression  fit  of  LC50  as  a  function  of  Vmtio 
(with  a  p-value  <  0.001)  qualitatively  suggests  that 
species  respiratory  geometry  may  play  a  factor  in 
predicting  chlorine  inhalation  toxicity.  However, 
due  to  the  database  composition  (where  Vratl0  is 
inadvertently  correlated  with  M),  a  quantitative 
conclusion  cannot  be  reached  at  this  time. 

Based  upon  the  final  weight  assignments  from  the 
IRLS  procedure  for  the  LD50  fit  (see  Appendix  F), 
the  least  valuable  data  are:  the  5-min  and  10-min 
rat  LCT50  values  from  Zwart  and  Wourterson;78  the 
30-min  mouse  LCT50  value  from  Schlagbauer  and 
Henschler;81  and  the  30-min  sheep  LCT50  values 
from  Batchinsky  et  a/.87  All  of  these  received 
weight  values  less  than  0.65.  The  most  valuable 
data  are:  the  18. 5-min  goat  LCT50  value  from 
McElroy  et  al.; 74  and  the  10-min  mouse  LCT50 
value  from  Bunting.79  Both  are  close  to  the 
maximum  value  of  2  and  are  greater  than  1.95  in 
value. 
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Figure  7-2.  Comparison  of  human  estimates  (military  personnel)  from  various 
model  fits  in  the  present  study 


7.3  Toxic  Load  Exponent  and  Time- 
Concentration  Relationship 

7.3.1  Estimation  from  Allometric  Modeling 
of  Historical  Database 

Using  eq  3  and  eq  10,  a  reformulation  is  used  to 
derive  the  toxic  load  exponent. 

log(LDso)  =  log(LCTS0)  +  log(UM)  =  k„  +  k„,  log(A/)  + 
k,  log(7) 

log(C)  +  log(7)  =ka  +  km  log (M)  -  \og(VM)  +  k,  log(7) 
log(C)  +  log(U  -  k,  log (T)  =  k0  +  km  log (M)  -  \og(VM) 
log(C)  +  log(7)  -  k,  log(r)  =  ka  +  km  log (M)  -  log(  VM) 
log(C)  +  (1  -  kt)  log(r)  =  k0  +  km  log(Af)  -  log(  VM) 

(1/[1  -  k ,])  log(C)  +  log(7)  =  (1/[1  -  k,])  (k0  +  km  logfifr] 

-  logtUd) 

l0g(Ca,[1-M:i)  +  log(r)  =  (1/[i  _  kt])ko  +  km  iog[M] 

-  log[  VM  ]) 

log(C(1/[1-A']))  =  (1/[1  -k,])ko  +  km  log[M]  -logrM]) 

Written  in  this  form,  the  toxic  load  exponent 
equals  1/[1  -  £,].  The  relationship  between  the 
concentration  and  the  duration  of  exposure  is  given 
by  this  last  equation. 


For  the  LD50  fit  (eq  19),  substituting  the  value  of 
kt  =  0.62960;  the  toxic  load  exponent  becomes 
1/(1  -  0.62960)  or  approximately  2,70. 

For  a  healthy  human  being,  the  minute-volume  for 
light  activity  is  assumed  to  equal  0.015  mVmin, 
and  the  standard  military  mass  is  70  kg.7 
The  right-hand  side  of  the  equation  becomes: 

( l/[  1  -  k,] )  x  {k„  +  km  log[M]  -  log [MV\)= 

(2.69978)  x  [0.5147  +  0.86908  log(70)  -  log(0.015)] 

=  10.64293 

These  quantities  produce  the  relationship  (shown 
also  in  Figure  7-3): 

log(C2  77)  =  10.64293,  resulting  in  the  toxic 
load  equation: 

C2  7T=  4.39  x  1010  (22) 

From  the  linear  regression  fit  for  eq  19,  an  estimate 
can  be  obtained  for  the  standard  error 
(SE  =  0.05939)  for  the  fitted  coefficient,  kt  (see 
Appendix  F).  It  is  possible  to  construct  95% 
confidence  intervals  for  kt  from  its  SE 
(kt±  (1.96)SE).  The  95%  confidence  intervals  for 
the  toxic  load  exponent  ( n )  can  be  generated  by 
substituting  the  upper  and  lower  limits  for  kt  into 
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Time  (minutes) 

Figure  7-3.  Comparison  of  human  estimates  from  allometric  fit  to  previous 
human  estimates  and  experimental  mammalian  data 


the  relationship,  { 1/[1  -  A',] } .  The  fitted  value  for  kt 
and  the  calculated  value  for  n  are  listed  in 
Table  7-4,  along  with  their  95%  confidence 
intervals. 

Since  the  95%  confidence  bounds  of  the  toxic  load 
exponent  do  not  include  the  value  of  one,  Haber’s 
Rule — where  n  equals  1  or,  equivalently,  that  the 
value  of  a  LCT50  (or  ECT50)  is  constant  with 
respect  to  exposure  duration — is  not  applicable  for 
chlorine.  Therefore,  the  true  relationship  between 
chlorine  concentration  and  time  of  exposure  is 
appropriately  given  by  the  toxic  load  model 
(L(C"7)50  =  constant).  However,  the  precision  for  n 
is  very  low,  with  the  95%  confidence  intervals  for 
the  toxic  load  exponent  ranging  from  1.7  to  6.0.  A 
possible  explanation  for  this  is  the  relatively 
narrow  range  of  exposure  durations  that  have  been 
investigated  historically,  with  the  bulk  of  the 
exposure  durations  ranging  from  1 0  min  to  60  min. 


Table  7-4.  Toxic  load  exponent  estimate  from 
allometric  model  fit 


POINT 

ESTIMATE 

LOWER 
BOUND  (95%  Cl) 

UPPER 

BOUND  (95%  Cl) 

kt  (coefficient  of 
duration  time) 

0.63 

0.43 

0.83 

n  (toxic  load 
exponent) 

2.70 

1.74 

5.97 

Also,  the  presence  of  random  error  between  the 
various  studies  could  decrease  the  lack  of  precision 
in  the  toxic  load  exponent.  Thus,  another  approach 
was  investigated  in  the  development  of  an  estimate 
for  the  toxic  load  exponent  in  a  human  toxicity 
relationship  was  investigated  (see  Section  7.3.2). 
7.3.2  Estimation  from  Individual  Studies 

If  random  error  among  several  studies  is  an  issue 
in  estimating  the  toxic  load  exponent,  then  a 
review  of  individual  studies  that  investigated 
several  exposure  durations  can  be  helpful.  This 
was  the  approach  taken  by  ten  Berge  and  van 
Heemst51  and  ten  Berge  et  al54  However,  only  two 
studies  represented  in  Table  5-1  (page  5-2) 
involved  exposures  at  several  exposure  durations: 
Bitron  and  Aharonson66  and  Zwart  and 

78 

Woutersen. 

Bitron  and  Aharonson66  exposed  male  mice  to  a 
range  of  exposure  durations  at  two  constant  vapor 
concentrations  (durations  of  13  min  to  160  min  for 
170  ppm,  and  6  min  to  30  min  for  290  ppm).  For 
each  experimental  run,  14  mice  were  exposed 
(with  each  mouse  in  his  own  chamber  connected  to 
a  central  air/chlorine  vapor  source).  There  were  16 
runs  using  170  ppm  and  18  runs  using  290  ppm, 
for  a  grand  total  of  476  mice.  The  dataset  is  well 
balanced,  with  a  good  distribution  of  percent 
response  values  that  cover  at  least  5%  to  95% 
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range  for  each  concentration.  A  value  for  the  toxic 
load  exponent  was  not  originally  reported,  but  ten 
Berge  and  van  Heemst51  subsequently  calculated  a 
value  of  3.5  [with  95%  confidence  interval  (Cl)  of 
2.5  to  4.5]  from  the  data  of  Bitron  and  Aharonson. 

Zwart  and  Woutersen78  exposed  both  male  and 
female  animals  to  several  concentrations  at  four 
exposure  durations  (5  min,  10  min,  30  min,  and 
60  min)  for  rats  and  at  two  durations  (10  min  and 
30  min)  for  mice.  Ten  animals  (five  of  each 
gender)  were  exposed  in  a  common  chamber. 
There  were  22  runs  with  the  rats  (220  rats  total) 
and  11  runs  with  the  mice  (100  mice  total). 
However,  they  did  not  break  down  their  response 
results  by  gender.  It  is  known  that  there  can  be 
gender  differences  in  the  toxic  response  of 
rodents52  to  toxicants  in  general.  Also,  their  dataset 
is  not  as  well  balanced  as  that  of  Bitron  and 
Aharonson,  particularly  for  the  rat.  The  5 -min  and 
10-min  durations  response  data  are  sparse  on 
responses  that  are  not  0%  or  100%,  and  the  rat 
LCT5o  values  at  5  min  and  10  min  are  higher  than 
the  value  at  30  min  (or  a  toxic  load  exponent  less 
than  one  for  the  range  of  5  min  to  30  min).  It  is 
interesting  to  note  that  Mannan27  does  not  mention 
the  results  of  Zwart  and  Woutersen  in  his  extensive 
review  of  the  existing  chlorine  toxicity  data. 
Considering  the  relative  merits  of  the  two  studies, 
the  data  of  Bitron  and  Aharonson  is  superior  to  that 
of  Zwart  and  Woutersen  for  estimating  the  toxic 
load  exponent. 

Response  results  as  a  function  of  chlorine  vapor 
concentration  and  exposure  duration  were 
extracted  from  the  graphical  representations  shown 
in  Bitron  and  Aharonson.66  A  multifactor  probit 
analysis  was  then  performed  on  the  data  using  the 
binary  logistic  regression  routine  in  Minitab" .  It 
was  found  that  the  toxic  load  exponent  equals  3.36 
with  95%  Cl  of  2.99  to  3.73.  This  confidence 
interval  is  considerably  narrower  than  was 
originally  calculated  by  ten  Berge  and  van 
Heemst.51  However,  there  was  statistically 
significant  lack  of  model  fit.  Two  outliers  were 
identified  as  having  a  high  leverage  on  the  final 
model  fit.  When  these  outliers  were  dropped  from 
the  analysis,  the  toxic  load  exponent  was  found  to 
equal  3.18  with  95%  Cl  of  2.90  to  3.46,  which  is 
not  drastically  different  from  the  model  fit  with  the 
outliers.  Also,  the  lack  of  model  fit  disappeared  per 
the  results  of  the  Hosmer-Lemeshow  lack  of  fit 


test.  See  Appendix  J  for  a  complete  description  of 
the  statistical  analysis. 

Thus,  an  examination  of  the  three  allometric-based 
toxicity  fits  (eqs  18,  20,  and  21)  shown  in  Figure 
7-2  finds  that  the  calculated  toxic  load  exponent 
from  Bitron  and  Aharonson  is  in  better  agreement 
with  the  exponent  values  from  eqs  18  and  20.  The 
fit  (eq  21)  using  the  minute  volume  ratio  (VM) 
produces  an  exponent  value  that  is  too  low. 

7.4  Human  Chlorine  Inhalation  Toxicity 
Estimates 

Three  parameters  are  needed  to  mathematically 
define  a  final  inhalation  toxicity  estimate  (as 
represented  in  eq  23  below):  (1)  a  probit  slope  to 
describe  the  range  of  individual  sensitivities  to  the 
toxicant  (kTr)',  (2)  the  median  lethal  dosage  (or 
toxic  load)  at  some  reference  exposure  duration  ( kQ 
when  Yn  is  set  equal  to  zero);  and  a  toxic  load 
exponent  (n)  to  describe  how  the  toxicity  changes 
with  exposure  duration. 

Yn  =  k0  +  kTL  log(C"7)  (23) 

In  the  following  two  sections,  the  derivation  of  the 
final  human  estimates  is  described  for  both  a 
military  subpopulation  and  the  general  population. 
These  are  shown  in  Figure  7-4  in  comparison  to 
previously  derived  estimates  and  the  existing 
mammalian  database.  The  final  military  sub¬ 
population  estimate  is  slightly  different  from  what 
was  derived  via  allometric  scaling  (eq  22)  for 
reasons  explained  in  greater  detail  below. 

7.4.1  Military  Subpopulation 

7.4.1. 1  Probit  Slope 

For  the  probit  slope,  a  weighted  average  from  the 
available  experimental  data  (see  Section  7.1.1)  for 
healthy  laboratory  animals  produced  a  value  of  8.0 
(rounded  from  7.96),  with  95%  Cl  of  7.2  to  8.7.  It 
is  assumed  that  this  is  an  appropriate  estimate  for 
healthy  (military)  human  subpopulation. 

7. 4.1. 2  Reference  Median  Lethal  Dosage 

For  the  median  lethal  toxic  load,  there  is  some 
leeway  for  a  recommended  value  due  to  the  large 
amount  of  random  error  present  among  the  various 
experimental  studies  (see  Table  5-1,  page  5-2).  In 
Figure  7-3,  the  allometric  fit  (eq  18)  is  shown  in 
comparison  to  previously  derived  estimates  by 
other  researchers.  The  allometric  fit  is  higher  than 
all  of  the  previous  estimates,  with  those  of  ten 
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Berge  and  van  Heemst51  and  the  Rijnmond 
Report59'64  being  the  closest.  Eisenberg  et  al T7  and 
Perry  and  Articola63  can  be  dropped  from  consider¬ 
ation  due  to  the  faulty  assumptions  used  in  their 
derivations. 27'34,39,40’51  There  are  two  reasons  for  the 


difference  between  eq  18  and  ten  Berge  and  van 
Heemst  and  the  Rijnmond  Report.  First,  eq  18  is 
meant  for  use  with  a  healthy  (military) 
subpopulation,  while  the  other  two  are  for  the 
general  population.  This  issue  is  further  addressed 
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a.  Median  lethal  dosage  (LCT50)  versus  exposure  duration  (T) 
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Figure  7-4.  Comparison  of  human  estimates  (military  and  general)  with  previous 
human  estimates  and  experimental  mammalian  data 
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in  Section  7.4.2.  Second,  eq  18  is  based  on  the  fact 
that  both  LD50  (absolute)  and  LC50  increase  as  a 
function  of  increasing  species  body  mass  (as 
demonstrated  by  statistical  analysis  in  the  present 
study).  Both  ten  Berge  and  van  Heemst  and  the 
Rijnmond  Report  assumed  that  a  human  estimate 
can  be  modeled  on  an  average  for  all  mammals 
(see  Section  3.2).  So,  as  a  starting  point  for  a 
human  general  population  estimate,  the  human 
healthy  (military)  LCT50  for  a  2-min  exposure  was 
taken  from  eq  18,  or  13,500  mg-min/m3. 

7. 4.1. 3  Toxic  Load  Exponent 

For  the  toxic  load  exponent,  past  approaches  have 
been  somewhat  subjective,  ten  Berge  and  van 
Heemst51  took  an  average  of  Bitron  and 
Aharonson66  (n  =  3.5)  and  Anglen67  (n  =  1.9)  to 
arrive  at  2.75.  This  may  be  a  case  of  a  correct 
answer  being  obtained  by  incorrect  methods. 
Bitron  and  Aharonson  was  a  lethality  study,  while 
Anglen  investigated  chlorine  vapor  concentrations 
that  cause  olfactory  irritation.  The  toxic 
mechanisms  are  likely  different  between  lethality 
(from  irritation  of  the  lungs)  and  nuisance  effects 
(due  to  local  irritation  of  the  nasal  mucous 
membranes).  Thus,  the  toxic  load  exponent  values 
for  the  two  phenomena  are  likely  not  equal,  and  so, 
the  use  of  an  average  of  the  two  is  not  the  proper 
approach  to  obtain  an  estimate  for  the  lethal  toxic 
load  exponent. 

Harris  and  Moses  (1983),  in  their  subjective  review 
of  existing  animal  data,  concluded  that  Coast 
Guard  reports57'63  have  the  toxic  load  exponent 
correct  at  2.75.  However,  it  is  not  readily  evident 
how  the  Coast  Guard  arrived  at  their  estimate, 
though  it  appears  that  they  performed  an  empirical 
fit  of  previous  human  estimates  instead  of  basing  it 
on  the  available  mammalian  data. 

Withers  and  Lees39  based  their  estimate  (n  =  2)  on 
an  examination  of  the  LCT50  values  for  several 
exposure  durations  reported  by  Weedon  et  a/.88  and 
Bitron  and  Aharonson.66  Others  have  used  the 
same  value,  though  for  different  reasons.49'61 
Withers  and  Lees  did  note  a  major  problem  with 
Weedon  et  al.  (the  gas  until  death  exposure 
protocol),  which  made  its  use  less  than  desirable. 
Also,  they  had  a  concern  about  the  unusually  low 
proportion  of  acute  deaths  in  Bitron  and 
Aharonson.  However,  Withers  and  Lees  felt  that 
these  were  the  only  two  studies  available  from 
which  to  justify  the  effect  on  exposure  duration  on 


chlorine  toxicity.  A  major  difference  between 
Weedon  et  al.  and  Bitron  and  Aharonson  is  the 
number  of  animals  used  in  each  study:  Weedon  et 
al.  used  eight  rats  and  four  mice  for  each  LCT50 
estimate  (for  a  total  of  16  rats  and  8  mice);  and 
Bitron  and  Aharonson  used  476  mice  (roughly 
equally  divided  between  their  two  LCT50 
estimates).  Withers  and  Lees  took  the  six  LCT50 
values  from  the  two  reports  and  calculated  values 
of  L(CT<1/n))  for  several  values  for  (1/n) — 0.4,  0.5, 
and  0.6.  They  concluded  that  a  value  of  0.5 
produced  L(CT(1/n))  values  that  were  in  the  best 
agreement  with  one  another.  A  subsequent 
reexamination  by  this  study  of  their  approach 
found  that  an  exponent  value  of  0.33  (or  n  =  3) 
works  best  for  the  Bitron  and  Aharonson  values 
and  an  exponent  value  of  0.55  (or  n  =  1.82)  for 
Weedon  et  al.  values.  All  other  factors  being  equal, 
more  weight  should  have  been  given  by  Withers 
and  Lees  to  the  Bitron  and  Aharonson  value,  since 
it  is  based  upon  the  response  of  476  mice  versus 
the  24  rodents  used  in  Weedon  et  al.  If  a  weighted 
average  is  taken  of  0.33  and  0.55  (using  the 
number  of  animals  for  the  weights),  a  value  of  0.34 
(or  n  =  2.94)  is  obtained. 

The  MHAP  also  concluded  that  2.75  was  a 
reasonable  estimate  for  the  toxic  load  exponent  in 
their  development  of  a  threshold  severe  effects 
level.34  They  reasoned  that  the  evidence  for  this 
value  is  much  stronger  than  for  the  lower  values 
derived  by  others  (such  as  2.0  by  Withers  and 
Lee39'40).  They  believed  that  the  value  of  2.75  is 
advantageous  because  it  results  in  conservative 
assessments  for  short  exposures  (less  than  10  min) 
where  there  is  considerable  uncertainty  in  the  dose- 
response  relationship. 

In  summary,  this  study  is  in  agreement  with  both 
ten  Berge  and  van  Heemst51  and  Withers  and 
Lees39  on  the  importance  of  Bitron  and 
Aharonson.66  The  results  of  Bitron  and  Aharonson 
support  a  higher  value  (in  the  range  of  3  to  3.5)  for 
the  toxic  load  exponent  than  has  been  used  in  the 
previous  human  estimates.  Of  the  available 
experimental  studies,  Bitron  and  Aharonson  have 
the  best  set  of  lethality  data  involving  multiple  ex¬ 
posure  durations.  Yet,  this  is  only  one  study,  which 
has  not  yet  been  reproduced. 

Considering  all  of  this,  some  caution  is  in  order 
with  respect  to  recommending  a  toxic  load 
exponent  value,  since  setting  the  toxic  load 
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exponent  value  too  high  will  underestimate  the 
toxicity  for  longer  exposure  durations.  The  range 
of  reasonable  values  to  choose  from  can  be  defined 
on  the  low  end  by  the  allometric  fit  from  this  study 
(n  =  2.70  from  eq  18)  and  on  the  upper  end  by  this 
study’s  estimate  of  n  from  the  data  of  Bitron  and 
Aharonson  (n  =  3.35).  Thus,  for  a  human  estimate, 
a  toxic  load  exponent  of  2.75  was  chosen  (as  a 
conservative  estimate)  for  generating  a  final  human 
(military)  toxicity  time  relationship.  This  favors 
(for  the  moment)  the  bulk  of  the  historical  database 
(Table  5-1)  and  previous  human  estimates  for  the 
exponent  value.34,51,57,59’63,64  However,  this  toxic 
load  exponent  should  be  reevaluated  in  the  light  of 
any  applicable  future  mammalian  studies. 

7. 4.1. 4  Human  (Military)  Lethality 
Relationship  from  Allometric  Fit 

Based  upon  the  discussion  of  the  previous  sections, 
the  healthy  (military)  human  toxicity  estimate 
based  upon  allometric  scaling  (Section  7.2)  is 
defined  by  eq  24  (see  also  Figure  7-4a): 

Yn  =  (-31 .52  +  (2.91 )  log(C2  75  T) 

or  (24) 

L(C2  75T)50  =  6.79  x  1010 

where  C  is  in  mg/m3  and  T  is  in  minutes.  In  Table 
7-5,  a  listing  is  given  of  LC5o  and  the  LCT50  values 
for  select  exposure  durations. 

7.4.2  General  Population 

Estimates  for  the  general  population  were  generat¬ 

Table  7-5.  LCT50  and  LC50  values  from  military 
chlorine  toxicity  estimate  (eq  24) 


ed  from  the  allometric  scale-based  estimate  for  the 
military  subpopulation  (eq  24)  via  the  method  of 
Crosier1  (see  Section  2.4).  Using  this  method,  the 
following  values  were  obtained: 

Probit  slope  (general  population)  ( kc )  =  6.0 

LCT50  for  2  min  (general  population)  =  9500  mg- 
min/nr1 

For  the  toxic  load  exponent,  mathematical 
considerations  require  that  both  a  subpopulation 
and  the  general  population  must  have  identical 
toxic  load  exponent  values  (e.g.,  differing  values 
for  n  can  produce  the  paradoxical  situation  where 
the  median  lethal  dosage  for  the  general  population 
could  exceed  that  of  the  military  subpopulation). 
Putting  these  three  parameters  together  produces 
the  probit  relation  for  the  general  population: 

Yn  =  (-22.698)  +  (2.18)log(C2757 ) 

or  (25) 

L(C2  75T)5o  =  2.58x1010 

where  C  is  in  mg/m3  and  T  is  in  minutes.  A  listing 
is  given  of  LC50  and  the  LCT50  values  for  select 
exposure  durations  in  Table  7-6  (see  also 
Appendix  H). 

7.5  HPAC  Modeling  Results 

To  provide  a  sanity  check  for  the  new  human 
estimates,  a  series  of  modeling  runs  were 
conducted  using  the  Hazard  Prediction  and 
Assessment  Capability  (HPAC)  model  (see 

Table  7-6.  LCT50  and  LC50  values  from  general 
population  chlorine  toxicity  estimate  (eq  25) 


EXPOSURE 
DURATION  (min) 

MILITARY  SUBPOPULATION 

EXPOSURE 
DURATION  (min) 

WHOLE  POPULATION 
SINGLE  TRUNCATION 

Probit  Slope  (fc)  =  8.0 

Probit  Slope  (fc)  =  6.0 

LCT5o(mg-min/m3) 

LC50  (mg/m3 ) 

LCT5o(mg-min/m3) 

LC50  (mg/m3) 

2 

13500 

6750 

2 

9500 

4750 

10 

37600 

3760 

10 

26400 

2640 

30 

75700 

2520 

30 

53200 

1780 

60 

118000 

1960 

60 

82700 

1380 

120 

183000 

1520 

120 

129000 

1080 

240 

284000 

1180 

240 

200000 

830 

360 

368000 

1020 

360 

259000 

720 

480 

442000 

920 

480 

311000 

650 

960 

687000 

720 

960 

483000 

500 
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Appendix  I).  For  a  typical  large-scale  accidental 
release  of  chlorine,  vapor  concentration  versus 
time  profiles  were  generated  for  several  locations 
downwind.  Then  several  human  (general 
population)  toxicity  estimates  were  used  to 
estimate  the  probability  of  an  exposed  individual 
dying  should  they  be  exposed  to  these 
concentration-time  profiles. 

7.5.1  HPAC  Model  Inputs  and  Outputs 

The  scenario  modeled  was  for  a  catastrophic 
release  of  50  tons  (-45,500  kg)  of  chlorine  liquid. 
Ninety  tons  is  the  typical  amount  for  a  standard  rail 
tank  car.  Flowever,  a  release  of  50  tons  was  used  as 
input  into  F1PAC,  since  this  is  the  largest  release 
reported  by  Mannan27  in  their  review  of  chlorine 
industrial/transportation  accidents.  Literature 
commentary  on  the  downwind  location  of  fatalities 
(see  Section  4)  uses  this  accident  database  as  its 
reference.  Thus,  modeling  results  for  the  50-ton 
release  were  used  for  comparison  to  the  different 
toxicity  methodologies. 

Model  inputs  were  kept  fairly  simple  for  these 
simulations.  Flat  terrain  was  assumed. 
Meteorological  conditions  were  kept  at  a  steady- 
state,  fixed-wind  condition  (same  wind  direction 
and  wind  speed  assumed  through  the  simulation). 
The  chlorine  release  was  assumed  to  be  a 
catastrophic,  instantaneous  release  of  the  liquid. 
F1PAC  make  estimates  of  the  amount  of  chlorine 
that  initially  pools,  but  all  liquid  evaporated  very 
quickly.  Since  meteorological  conditions  can 
produce  significant  differences  in  hazard 
prediction  results,  three  different  weather 
conditions  were  modeled.  These  three  conditions 
were: 

•  Low  wind  (1  m/s),  clear  sky,  nighttime  (2  a.m.), 
“stable”  condition  (Pasquill90  Stability  Category 
F). 

•  Moderate  wind  (5  m/s),  cloudy  day,  “neutral” 
condition  (Pasquill  Stability  Category  D). 

•  Low  wind  (2  m/s),  clear  sky,  daytime  (2  p.m.), 
“unstable”  condition  (Pasquill  Stability  Category 
B). 

The  F1PAC  model  provided  numeric  output  of 
concentration  time  history  for  each  model  run  at 
downwind  distances  of  0.2  km,  0.5  km,  1.0  km,  1.5 
km,  and  2.0  km.  The  concentration  time  histories 
were  then  numerically  integrated  using  several 
different  human  toxicity  estimates  from  Table  7-2 


and  eq  25  to  calculate  the  probability  of  lethality  at 
each  downwind  sampler  location. 

In  the  course  of  this  exercise,  it  was  discovered 
that  HPAC  cannot  properly  display  LTLXx 
contours  in  its  graphical  output  though  it  can 
properly  calculate  exposure  toxic  loads  (see 
Appendix  I  for  a  more  detailed  discussion).  This  is 
why  a  numeric  output  of  the  concentration-time 
history  was  subsequently  used  for  this  section. 

7.5.2  Sensitivity  of  HPAC  Model 
Predictions  to  Changes  in  Toxicity 
Estimate  Parameters 

Not  surprisingly,  the  relationship  of  the  probability 
of  lethality  as  a  function  of  downwind  distance  is 
dependent  on  the  toxicity  estimate  used.  The  nature 
of  this  relationship  is  demonstrated  in  Figure  7-5 
using  the  HPAC  concentration-time  profile 
generated  using  nighttime  conditions  (Pasquill 
Stability  Class  F).  Four  curves  were  generated 
using  a  two  by  two  matrix  of  kc  (3  and  6)  and  n 
(1.85  and  2.75)  values.  All  four  curves  share  an 
identical  LC50  value  of  4750  mg/m3  at  2  min. 
Equation  25  (for  the  general  population)  is 
represented  by  the  curve  labeled  kc  =  6  and  n  = 
2.75. 

A  decrease  in  the  toxic  load  exponent  value  (with 
everything  else  constant)  shifts  the  line  to  the  right, 
with  the  new  curve  still  being  parallel  to  the 
original  line  (when  plotted  on  a  probability  versus 
log(distance)  scale).  A  decrease  in  the  probit  slope 
(concentration),  kc,  (with  everything  else  constant) 
rotates  the  new  line  counterclockwise  relative  to 
the  original  line.  Changing  both  kc  and  n  both  shifts 
and  rotates  the  new  line  compared  to  the  original. 
Since  a  probit  slope  change  rotates  the  line,  the 
differences  in  downwind  distances  between  the  old 
and  new  lines  is  a  function  of  the  lethality 
probability,  with  the  difference  increasing  as  you 
move  closer  or  further  from  the  point  of 
intersection  between  the  two  lines. 

7.5.3  Comparison  of  Human  Toxicity 
Estimates  in  Predicting  Lethality 
Probabilities  from  HPAC  Model  Predictions 

In  Figure  7-6,  the  probability-distance  profile 
based  upon  eq  25  is  compared  to  profiles  generated 
using  previously  derived  human  estimates  (see 
Table  7-2)  for  the  HPAC  concentration-time 
profile  generated  for  nighttime  conditions  (Pasquill 
Stability  Class  F).  For  probabilities  below  10%,  the 
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Figure  7-5.  Probability  of  lethality  versus  downwind  distance  as  a  function  of 
toxic  load  expression  (Pasquill  Stability  Category  F) 


downwind  distance  predicted  by  eq  25  is  less  than 
what  is  predicted  by  Withers  and  Lees,40  Rijnmond 
Report/Harris  and  Moses59'64  and  ten  Berge  and 
van  Heemst.51  Distances  predicted  by  Haber’s  Rule 
(using  a  LCT50  of  9300  mg-min/m3  and  a  probit 
slope  of  6)  were  also  calculated.  Considering  the 


historical  record  (see  Section  4),  the  downwind 
distances  predicted  by  Haber’s  Rule  are  very 
unrealistic  (i.e.,  a  20%  chance  of  lethality  at  a 
downwind  distance  of  2  km). 


Figure  7-6.  Probability  of  lethality  versus  downwind  distance  for  several  human 
chlorine  lethality  estimates  (Pasquill  Stability  Category  F) 
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Figure  7-7.  Probability  of  lethality  versus  downwind  distance  for  several  human 
chlorine  lethality  estimates  (Pasquill  Stability  Category  B) 


Of  the  other  three  studies,  the  downwind  prediction 
curve  from  the  Rijnmond  Report  comes  the  closest 
in  location  and  orientation  to  that  of  eq  25.  The 
probit  slope  and  toxic  load  exponent  values  from 
the  Rijnmond  Report  (km  =  1-89  and  n  =  2.75)  are 


very  similar  to  that  of  eq  25  (km  =  2.18  and  n  = 
2.75).  The  difference  between  the  two  curves  is 
mainly  due  to  differences  in  the  median  lethal  toxic 
load  used  in  the  two  toxicity  estimates.  Both 
Withers  and  Lees  and  ten  Berge  and  van  Heemst 


Downwind  Distance  (km) 

Figure  7-8.  Probability  of  lethality  versus  downwind  distance  for  several  human 
chlorine  lethality  estimates  (Pasquill  Stability  Categories  B  and  F) 
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have  smaller  probit  slope  values  than  eq  25  and  the 
Rijnmond  Report,  and  thus,  their  predictions 
curves  are  rotated  counterclockwise  relative  to 
eq  25  and  the  Rijnmond  Report  (per  the 
observations  of  the  sensitivity  study  in  the  previous 
paragraph).  In  Figure  7-7,  the  same  toxicity 
relationships  used  for  Figure  7-6  are  used  for  the 
HPAC  concentration-time  profile  generated  for 
day  time  conditions  (Pasquill  Stability  Class  B). 

In  Figure  7-8,  the  previous  plots  for  eq  25,  ten 
Berge  and  van  Heemst  and  the  Rijnmond  Report 
are  shown  for  both  day  and  nighttime  conditions 
[plots  for  overcast  conditions  (Pasquill  Stability 
Class  D)  are  very  similar  to  those  of  nighttime].  As 
previously  mentioned  in  Section  4,  most  fatalities 
from  industrial/transportation  accidents  involving 
chlorine  have  occurred  with  250  m  of  the  point  of 
release  (even  when  considering  large  20-ton  to 
50-ton  releases),  and  the  maximum  distance  is 
400  m  (with  one  exception).  So,  any  proposed 
general  population  toxicity  estimate  should 
produce  downwind  prediction  curves  that  are 
consistent  with  the  historical  record  (assuming  that 
the  ATD  model  is  adequately  predicting  vapor 
dispersion).  Under  daytime  conditions,  the  plots 
for  eq  25,  ten  Berge  and  van  Heemst  and  the 
Rijnmond  Report  are  consistent  with  the  historical 
record.  However,  for  nighttime  conditions,  all 
three  toxicity  estimates  still  produce  significant 
lethality  probabilities  at  400  m  downwind  (from 
40%  to  75%).  From  this  point  downwind, 
probabilities  from  both  eq  25  and  the  Rijnmond 
Report  fall  off  quickly  as  a  function  of  distance 
relative  to  that  predicted  by  ten  Berge  and  van 
Heemst.  Is  possible  that  deaths  have  not  been  seen 
at  500  m  (despite  the  high  predicted  probabilities) 
due  to  other  factors  (i.e.,  more  time  available  for 
escape/sheltering  in  place  at  the  longer  downwind 
distances,  nighttime  accidents  may  not  be  that 
frequent  relative  to  daytime,  not  all  releases  are 
50  tons,  etc.).  For  distances  further  than  500  m,  the 
10%  lethal  probability  is  reached  at  500  m  with 
eq25,  650  m  with  the  Rijnmond  Report,  and 
800  m  with  ten  Berge  and  van  Heemst.  At  1  km, 
there  is  still  a  3%  lethal  probability  with  ten  Berge 
and  van  Heemst,  while  the  other  two  are  both 
below  0.2%.  This  greater  distance  on  the  part  of 
ten  Berge  and  van  Heemst  is  due  primarily  to  its 
shallower  probit  slope  ( kc  =  3.2  versus  5.19  and  6 
for  the  other  two  toxicity  estimates). 


7.5.4  Comparison  of  HPAC  Modeling 
Results  to  that  of  Franks  et  al.  (1996) 

Franks  et  al ,49  used  a  ATD  model  to  evaluate  the 
downwind  hazard  from  various  chlorine  release 
scenarios  (see  also  Section  4).  For  puff  releases, 
they  investigated  puff  sizes  ranging  from  500  kg  to 
5000  kg  [or  0.5  metric  tons  (M/T)  to  5  metric  tons] 
in  size,  using  a  human  toxicity  estimate  from  the 
Health  and  Safety  Executive  (HSE)  (UK).61  The 
toxicity  estimate  was  based  on  the  mouse  30-min 
LC50  value  of  Schlagbauer  and  Henschler,81  a 
probit  slope  ( kc )  of  6.67  (base  10),  and  a  toxic  load 
of  2  to  extrapolate  to  other  exposure  durations.  The 
LCT5o  as  a  function  of  T  for  this  relationship  is 
parallel  to  that  of  Withers  and  Lees  (see  Figure 
la)39,40  with  the  LCT50  of  Withers  and  Lees  being  a 
factor  of  1.6  higher.  The  underlying  assumption  of 
the  HSE  human  estimate  is  that  man  is  equivalent 
to  the  most  sensitive  mammal — the  mouse  in  this 
case. 

The  largest  puff  (5  metric  tons)  investigated  by 
Franks  et  al.  is  about  a  factor  of  1 0  lower  than  the 
50-ton  release  used  for  the  HPAC  runs  described 
above.  Even  so,  their  5-metric  ton  release  has  the 
LC50  extending  from  1400  m  to  1800  m  downwind. 
For  the  worse  case  scenario  in  the  HPAC  runs  (see 
Figure  7-6  through  Figure  7-8),  the  furthest  the 
LC50  extended  was  750  m  using  the  toxicity 
estimate  of  Withers  and  Lees  (if  the  unrealistic 
Haber’s  Rule  is  ignored). 
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8.0  Discussion/Conclusions 

The  development  of  a  human  chlorine  toxicity 
estimate  has  been  hampered  by  the  lack  of 
appropriate  and  accurate  toxicity  data.  As  a  result, 
various  investigators  have  had  to  resort  to  various 
degrees  of  subjective  “hand-waving”  to  produce 
appropriate  human  toxicity  estimates. 
Some39'40,51'59,64  have  been  more  successful  than 
others.57'61,63  The  present  study  improved  upon  the 
previous  work  by  applying  novel  approaches  (i.e., 
allometric  modeling,  weighted  average  of 
experimental  probit  slopes,  statistical  analysis  of 
original  response  data,  mathematical  modeling  of 
relationship  between  subpopulation  and  the  general 
population,  etc.)  to  the  problem  and  introducing 
previously  unknown70'74/underanalyzed74'84,85  mam¬ 
malian  lethality  data.  However,  even  with  these 
additional  objective  methods,  the  final  human 
estimate  is  only  as  good  as  the  underlying  original 
mammalian  data  that  it  is  based  upon.  Because  of 
significant  shortcomings  in  the  mammalian 
chlorine  lethality  data,  being  dogmatic  about  the 
final  human  estimate  should  be  avoided.  Thus,  the 
human  estimates  (for  both  military  and  general 
population)  developed  in  the  present  study  are 
offered  as  a  qualitative  (rather  than  quantitative) 
improvement  over  estimates  from  previous 
investigators.39,40,51'57,59,63,64  The  following  is  a  brief 
review  of  the  salient  issues  addressed  in  the  present 
study  and  how  they  differentiate  the  recommenda¬ 
tions  of  the  present  study  from  those  of  the 
previous  efforts. 

8.1  Variability  in  Human  Response 

It  has  been  recognized  by  many  that  the  general 
human  population  is  likely  to  have  more  variance 
in  its  response  to  chlorine  inhalation  than  healthy 
laboratory  animals.1,25,27, 33,34, 38-40, 45,51,59,60,, 64  The 

problem  has  been  how  to  best  extrapolate  probit 
slope  values  from  a  laboratory  animal  to  a  general 
human  basis. 

In  the  present  study,  more  statistical  rigor  was 
employed.  This  was  done  by  first  using  probit 
analysis  on  the  original  response  data  from  the 
experimental  studies  to  generate  probit  slope 
values  and  associated  standard  errors.  Then,  a 
weighted  average  of  the  probit  slopes  was 
calculated,  and  this  was  assumed  to  be  the  best 
estimate  for  variability  in  the  response  among  test 
mammals.  It  was  reasonably  assumed  that  healthy 
humans  were  equivalent  in  their  variability  to  that 


of  healthy  laboratory  mammals.  An  estimate  for 
the  general  population  was  generated  by  using  a 
mathematical  model  of  the  relationship  between  a 
subpopulation  and  the  whole  population  (see 
Section  7.4). 1  Of  the  previous  estimates  for  the 
probit  slope  of  the  general  population,  the  estimate 
of  the  present  study  ( kc  =  6)  is  closest  agreement  to 
that  of  the  Rijnmond  Report  (kc  =  5.2).  The  other 
important  studies  (ten  Berge  and  van  Heemst  and 
Withers  and  Lees)  arrived  at  lower  (more  response 
variability)  probit  slope  values. 

8.2  Time  Dependence  of  Chlorine 
Inhalation  Toxicity 

Previous  researchers39,40,51  have  recognized  (and 
the  present  study  has  further  confirmed — see 
Sections  7.3.1  and  7.4. 1.3)  the  futility  of 
calculating  a  precise  estimate  for  the  toxic  load 
exponent  based  on  the  existing  lethality  database  in 
total.  Instead,  the  individual  findings  of  four 
experimental  studies66,67,78,88  have  been 
cited39,40,51,78  as  the  basis  for  a  human  lethality 
exponent  value.  Of  these  four,  the  present  study 
has  concluded  that  only  Bitron  and  Aharonson66  is 
suitable  for  use  in  developing  a  lethality  exponent 
value  (see  Section  7. 4. 1.3).  However,  the  findings 
of  Bitron  and  Aharonson  have  been  misapplied  to 
the  problem.39,40,51  A  proper  use  of  Bitron  and 
Aharonson  would  lead  to  a  toxic  load  exponent 
value  in  the  range  of  3  to  3.4.  Allometric  modeling 
and  considering  the  database  as  a  whole  support  a 
slightly  lower  value  of  about  2.75.  The  “truth” 
probably  lies  somewhere  between  the  two 
extremes.  At  present,  a  lower  value  (2.75)  is 
recommended  for  two  reasons:  (1)  there  is  no 
collaborating  study  confirming  Bitron  and 
Aharonson;  and  (2)  the  desire  to  avoid 
underestimating  toxicity  at  longer  exposure 
durations  by  using  too  high  a  value  for  the  toxic 
load  exponent. 

8.3  Median  Lethal  Dosage 

The  basic  philosophy  for  extrapolating  from 
mammal  to  human  with  respect  to  median  lethal 
dosage  has  been  to  assume  that  humans  are 
equivalent  to  the  average  mammal.39,40,51,59,64 
However,  it  has  been  recognized  that  the  LC50  does 
increase  as  the  species’  body  mass  increases, 
though  the  exact  reason  for  this  has  been 
debated.34,39  An  allometric  fit  of  the  data  in  the 
present  study  found  that  the  median  lethal  dosage 
was  slightly  dependent  on  the  body  mass  (with 
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statistical  significance),  with  both  the  nominal 
LD50  (absolute)  and  LC50  increasing  as  the  species’ 
body  mass  increases.  However,  the  large  amount 
of  random  error  present  in  the  mammalian 
database  (Table  5-1)  prevents  a  precise  allometric- 
based  extrapolation  from  this  database  to  arrive  at 
a  human  estimate.  Thus,  the  “true”  answer 
probably  lies  inbetween  the  two  philosophies. 

8.4  Final  Human  Lethality  Estimates 

Of  the  five  previous  human  estimates  reviewed 
(see  Table  7-2),  the  Rijnmond  Report64/Harris  and 
Moses59  comes  the  closest  to  the  general 
population  estimate  of  the  present  work  (eq  25)  in 
terms  of  downwind  hazard  predictions  (see  Section 
7.5.3).  This  is  driven  mainly  by  the  identical  toxic 
load  exponent  values  (2.75)  and  similar  probit 
slope  ( kc )  values  (6  for  the  present  study  versus  5.2 
for  the  Rijnmond  Report).  However,  in  terms  of 
the  median  lethal  dosage,  the  relationship  from  ten 
Berge  and  van  Heemst51  is  equivalent  to  that  of  the 
Rijnmond  Report. 

The  HPAC  modeling  runs  (Section  7.5)  confirm 
the  often  observed  disconnect  between  the 
historical  record  (Section  4)  and  the  overly 
pessimistic  predictions  from  ATD  models.27  There 
are  three  possible  explanations  for  this  conflict. 
Either  the  ATD  models  are  overpredicting  the 
downwind  transport  of  chlorine,  the  human 
lethality  estimates  are  oveipredicting  the  toxicity, 
or  a  combination  of  both  of  these  factors.  There  is 
a  temptation  to  be  biased  towards  using  larger 
LCT50  estimates  to  achieve  more  reasonable 
casualty  predictions  from  the  current  ATD  models. 
However,  future  improvements  in  the  ATD  models 
(and/or  the  knowledge  of  the  atmospheric 
chemistry  of  chlorine)  could  leave  such  a  biased 
lethality  estimate  “out  of  position.” 

For  the  three  important  toxicity  estimate 
parameters  (probit  slope  (kc),  toxic  load  exponent 
(n),  and  reference  median  lethal  dosage),  the 
general  population  estimate  from  eq  25  is  in  better 
agreement  with  the  Rijnmond  Report  than  with  the 
other  previous  human  estimates,  particularly  for  kc 
and  n.  The  constant  difference  in  LCT50  values  is 
due  to  differences  in  extrapolation  philosophies 
[average  mammal  (Rijnmond)  versus  allometric 
scaling  (present  study)].  The  LCT50  estimate  of  eq 
25  is  about  a  factor  of  1.5  higher  than  that  from  the 
Rijnmond  Report,  which  is  not  much  above  the 
random  noise  in  the  data.  The  true  three  parameter 


fit  for  general  population  toxicity  is  probably 
bracketed  by  eq  25  and  the  Rijnmond  Report. 

In  summary,  the  following  upper  and  lower  general 
population  estimates  for  chlorine  lethality  are 
presented  for  consideration.  The  upper  estimate  is 
defined  by  eq  25,  and  lower  estimate  by  a  modified 
form  of  the  Rijnmond  Report  expression  (see 
Table  4).  The  Rijnmond  Report  expression  was 
modified  by  changing  the  original  probit  slope  ( kc ) 
value  equal  to  the  slope  in  eq  25  (a  change  from 
5.2  to  6.0),  and  then  a  2-min  reference  dosage  of 
6500  mg-min/m3  (calculated  from  the  Rijnmond 
estimate)  was  used  as  the  basis  for  a  toxic  load 
expression  (with  no  change  in  the  value  of  n).  In 
comparison,  the  2-min  general  population  LCT50 
value  from  eq  25  equals  9500  mg-min/m3.  These 
estimates  are  shown  below  in  eq  25  and  eq  26  and 
in  tabular  form  in  Table  8-1. 


Upper 

estimate  YN  =  (-22.698)  +  (2.l8)log(c2757) 
(adjusted  or 

allometric  l(c2  75t)  =  2.58  xIO10 

fit)  V  ho 

Lower  YN  =  (-21.710)  +  (2.18)log(C27!T) 
estimate  or 

(modified  l(C2  75T)  =  9.09x109 

Rijnmond)  v  750 


(25) 


(26) 


To  evaluate  the  relative  merits  of  eq  25  and  eq  26, 
it  is  recommended  that  a  more  extensive  sensitivity 
study  with  ATD  models  be  conducted  (which  was 
beyond  the  scope  of  the  present  work).  If  ATD 
models  continue  to  overstate  the  hazard  (even 
when  using  the  higher  estimate  of  eq  25),  then  it  is 
very  likely  that  the  problem  lies  with  the 
algorithms  of  the  ATD  models  (and/or  inadequate 
knowledge  of  cloud  transport  or  chlorine 
atmospheric  chemistry)  rather  than  in  an 
inadequate  lethality  estimate.  A  general  population 
median  lethality  estimate  higher  than  eq  25  cannot 
be  justified  using  the  current  lethality  database 
(Table  5-1). 

An  important  perforce  assumption  behind  eq  24  to 
eq  26  is  that  the  predicted  toxic  response  is 
independent  of  fluctuations  around  the  mean  vapor 
concentration  value  in  the  concentration-time 
history  profile  (see  Section  2.2).  If  this  should  be 
shown  in  the  future  not  to  be  the  case,  it  is  likely 
(based  on  theory)  that  eq  24  to  eq  26  will 
increasingly  underestimate  the  toxicity  as  the 
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Table  8-1.  Final  general  and  military  populations  dosage  and  concentration  estimates 


MODIFIED  RIJNMOND 

GENERAL  POPULATION 

MILITARY  POPULATION 

TIME 

GENERAL  POPULATION 

EQUATION  25 

EQUATION  24 

(min) 

LCTso 

(mg-min/m3) 

LCso 

(mg/m3) 

LCso 

(ppm) 

LCTso 

(mg-min/m3) 

LCso 

(mg/m3) 

LCso 

(ppm) 

LCTso 

(mg-min/m3) 

LCso 

(mg/m3) 

LCso 

(ppm) 

2 

6500 

3250 

1120 

9500 

4750 

1640 

13500 

6750 

2320 

10 

18100 

1810 

630 

26400 

2640 

910 

37600 

3760 

1300 

30 

36400 

1210 

420 

53200 

1780 

610 

75700 

2520 

870 

60 

56600 

940 

330 

82700 

1380 

480 

118000 

1960 

680 

120 

88000 

730 

250 

129000 

1080 

370 

183000 

1520 

530 

240 

137000 

570 

200 

200000 

830 

290 

284000 

1180 

410 

360 

177000 

490 

170 

259000 

720 

250 

368000 

1020 

359 

480 

213000 

440 

150 

311000 

650 

220 

442000 

920 

320 

960 

330000 

340 

120 

483000 

500 

170 

687000 

720 

250 

magnitude  of  the  fluctuations  increases,  since  n  >  1 
for  chlorine.  Operationally,  fluctuations  usually 
become  less  pronounced  as  the  distance  increases 
from  the  point  of  release. 
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9.0  Recommendations 

To  better  refine  the  human  chlorine  lethality 
estimates,  possible  future  toxicity  studies  should 
concentrate  on  the  following  issues.  First,  there  is 
currently  only  one  study66  that  was  properly  and 
accurately  conducted  that  provides  a  defensible 
estimate  for  the  lethality  toxic  load  exponent  (n). 
The  reproducibility  of  this  result  should  be 
confirmed  (over  a  range  of  exposure  durations 
from  several  minutes  to  several  hours),  preferably 
in  multiple  mammalian  species.  Second,  the 
usefulness  and  appropriateness  of  allometric 
scaling  can  be  strengthened  by  choosing  larger 
species  (i.e.,  rabbit,  swine  or  goat)  for  the 
previously  mentioned  toxicity  time-dependency 
studies.  Lastly,  comprehensive  sensitivity  studies 
using  ATD  models  should  be  conducted  to 
evaluate  against  the  historical  record  casualty 
predictions  from  all  toxicity  estimate  candidates. 
Independent  of  the  recommendations  with  respect 
to  chlorine  toxicity,  steps  should  be  taken  to 
improve  the  capability  of  HP  AC  to  graphically 
display  LTLxx  contours  in  its  plume  plots  (see 
Appendix  I  for  a  detailed  discussion).  Also,  the 
effect  of  concentration  fluctuations  on  the  toxicity 
of  chemicals  as  a  function  of  the  toxic  load 
exponent  and  magnitude  of  the  fluctuations  needs 
to  be  investigated  via  mammalian  inhalation 
studies.  As  mentioned  in  Section  2.2,  there  is 
currently  no  experimental  toxicity  data  available 
for  guidance  on  this  issue. 
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APPENDIX  A:  PROBIT  ANALYSIS  EXAMPLE 


The  data  from  Silver,  McGrath,  and  Ferguson1  is 
used  below  (Table  A-l)  to  illustrate  the  applica¬ 
tion  of  probit  analysis  to  raw  quantal  data  in  order 
to  estimate  probit  slopes.  In  this  lethality  study, 
mice  were  exposed  to  chlorine  for  a  period  of 
1 0  min. 

A  probit  equation  is  now  achieved  through  the  use 
of  Minitab®  Statistical  Software.  The  number  of 
deaths  was  imputed  as  “successes,”  and  the  link 
function  utilized  was  the  “normit.”  This  normit 
function  assumes  the  conventional  normal  distribu¬ 
tion  for  the  common  logarithm  of  the  concentra¬ 
tion.  In  other  situations,  the  common  logarithm  of 
the  dosage  (concentration  x  time  of  exposure)  is 
also  modeled  by  the  normal  distribution,  and  the 
normit  is  used  as  the  link  for  statistical  analysis 
within  Minitab®.  Binary  logistic  regression  is  the 
specific  routine  utilized. 

The  equation  that  is  derived  is  in  the  form: 


(mL/m3)],  and  Z  is  the  Z-score  from  the  standard 
normal  distribution. 

Binary  Logistic  Regression:  Number 
Deaths,  Number  Trials  versus  Log(C) 

Link  Function:  Normit 
Response  information: 


Variable 

Value 

Count 

Number  D 

Success 

157 

Failure 

143 

Number  T 

Total 

300 

See  logistic  regression  table  (Table  A-2) 


Table  A-2.  Logistic  regression  table 


PREDICTOR 

COEF. 

SE 

COEF. 

Z 

P 

Constant 

-25.735 

3.477 

-7.40 

0.000 

Log(C) 

7.826 

1.052 

7.44 

0.000 

Z  =  A  +  B  log(C)  (A1) 

Where  B  is  the  probit  slope,  A  is  the  intercept,  C  is 
the  concentration  [milligrams  per  cubic  meter 


Table  A-1.  Chlorine  lethality  data  for  mice  from  Silver,  et  al.  (1942) 


TIME  (min) 

CONCENTRATION 

tmn/m3t 

NUMBER  OF 
TRIALS 

NUMBER  OF 
DEATHS 

i°g(C) 

log(T) 

10 

1100 

20 

2 

3.04139 

i 

10 

1590 

20 

9 

3.20140 

i 

10 

1590 

20 

5 

3.20140 

i 

10 

1690 

20 

1 

3.22789 

i 

10 

1830 

20 

8 

3.26245 

i 

10 

1850 

20 

9 

3.26717 

i 

10 

2000 

20 

3 

3.30103 

i 

10 

2050 

20 

12 

3.31175 

i 

10 

2060 

20 

11 

3.31387 

i 

10 

2160 

20 

14 

3.33445 

i 

10 

2300 

20 

15 

3.36173 

i 

10 

2360 

20 

20 

3.37291 

i 

10 

2440 

20 

8 

3.38739 

i 

10 

2520 

20 

20 

3.40140 

i 

10 

2580 

20 

20 

3.41162 

i 

A-1 


Log-Likelihood  =  -170.220 

Test  that  all  slopes  are  zero:  G  =  74.795,  DF  =  1, 

P-value  =  0.000 

Goodness-of-Fit  Tests: 


Method 

Chi-Square 

DF 

P 

Pearson 

58.241 

12 

0.000 

Deviance 

68.052 

12 

0.000 

Llosmer-Lemeshow 

22.634 

5 

0.000 

The  remainder  of  the  calculations  uses  the  equation 
produced  to  calculate  the  median  lethal  concentra¬ 
tion  (LCT).  This  is  accomplished  by  setting  the  Z- 
score  to  zero,  and  solving  for  “C.” 

Z  =  -25.735  +  7.826  log(LC50) 

0  =  -25.735  +  7.826  log(LC50) 

logCLCso)  =  25.735/7.826 

log(LC5o)  =  3.288398 

LC50  =  1943 

The  median  lethal  dosage  (LD)  is  then  estimated 
by  multiplying  the  median  lethal  concentration  by 
the  duration  of  exposure.  The  result,  in  units  of 
milligram-minutes  per  cubic  meter  (mg-min/m3),  is 
posted  to  the  data  table  the  main  text. 

LCT5o  =10  min  x  1943  mg/m3=  19430  mg- 
min/m3 
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APPENDIX  B:  BINARY  AND  ORDINAL  PROBIT  MODELS  AND  THE  METHOD  OF 
MAXIMUM-LIKELIHOOD  ESTIMATION— Douglas  R.  Sommerville 


B1.0  Introduction 

Traditionally,  median  effective  dosages  (ED)  are 
determined  via  the  use  of  probit  analysis.1'2  In 
conventional  probit  analysis  of  binary  response 
data,  two  parameters  are  estimated  simultaneously 
from  experimental  quantal  data  using  the  method 
of  maximum  likelihood  estimation  (MLE):3'8  the 
median  effective  stress,  p;  and  the  standard 
deviation  of  effective  stresses,  a.  In  toxicology,  the 
effective  stress  is  the  base- 10  logarithm  of  the 
effective  dosage.  Thus,  the  base- 10  logarithm  of 
the  median  effective  dosage,  logio(ECT50), 
corresponds  to  p,  while  the  probit  slope  equals  the 
inverse  of  a.  The  probit  model  can  also  be 
extended  for  use  with  ordinal  response  data 
[categorical  data  that  have  three  or  more  possible 
levels  with  a  natural  ordering  (example,  mild, 
moderate,  and  severe)].1,3  For  the  ordinal  probit 
model,  there  are  individual  p’s  for  each  category, 
but  the  a’s  for  each  category  are  assumed  to  equal 
each  other. 

The  efficiency  of  the  MLE  procedure  with  the 
probit  model  is  dependent  on  the  sample  size. 
Larger  sizes  provide  unbiased  and  minimum 
variance  estimates  of  both  p  and  a,  but  this  is  not 
the  case  for  small  sample  sizes.  It  has  been  shown 
that  when  solving  for  both  p  and  a  with  small 
sample  sizes  that  estimates  for  p  are  unbiased  (for 
all  practical  purposes),  but  estimates  for  a  are 
biased  (with  estimates  for  a  being  too  small  on 
average). 3,8,9  Furthermore,  the  probability  of  MLE 
solution  instability  increases  as  sample  size 
decreases  when  trying  to  solve  for  both  p  and  a. 
Thus,  for  small  sample  sizes,  a  more  pragmatic 
approach  is  commonly  taken  by  fixing  a  at  some 
set  value  (based  on  historical  knowledge  of  the 
system  under  study)  while  solving  for  p.  This  is  an 
underlying  principle  of  the  up-and-down  method 
for  estimating  median  effective  stresses/dosages.6 
The  following  are  examples  of  both  a  binary  and 
ordinal  probit  model  applied  to  the  male  pig  GB 
ordinal  data  (severe  effects  and  lethality)  for  ten 
minutes  exposures  from  the  present  study.  The 
probit  slope  (1/a  or  m)  was  held  constant 
throughout  the  computations. 


B2.0  Binary  Probit  Model 

B2.1  MLE  Algorithm  for  Binary  Probit 
Model 

For  each  trial  condition,  i,  there  is  a  likelihood,  Lj, 
of  the  observed  result  occurring: 

£, ■  =  P?  V  -  P,)'-*  (B1a) 

loge(G)  =  x,  [loge/?,]  +  («,-*;)  [loga(1 -/>,-)]  (Bib) 

where  p\  is  the  event  probability  for  test  condition 
i,  «,  is  the  number  of  independent  trials  under  the 
/-th  condition,  and  x,  is  the  number  of  successes  in 
The  likelihoods  for  all  test  conditions  are  then 
multiplied  together  to  arrive  at  the  likelihood,  L. 
The  values  of  the  p' s  that  are  most  supported  by 
the  quantal  data  are  the  values  for  which  L  is  the 
largest.  For  ease  of  calculation,  the  natural 
logarithm  of  the  likelihood  is  often  used. 

For  a  normal  distribution,  /;,  is  defined  by  the 
following  relations: 

Pi  =  \7'  f{Z)dZ  (B2) 

J  —00 


f(z) 


(B3) 


where  Z,  is  the  standard  normal  random  variable 
and  /(Z)  is  the  probability  density  function  (pdf)  of 
a  standard  normal  distribution.  In  toxicology,  the 
values  of  the  individual  pi’s  are  a  function  of  the 
applied  dosages,  (CT)h  used  in  an  experiment  and 
their  respective  distances  from  the  median 
effective  dosage,  ECT50.  This  is  reflected  in  the 
following  definition  of  Z,: 

Z  =  =  m  {log10(CT)i  -  log10(ECT50 )}  (B4) 

(7 

where  S;  is  the  applied  stress  for  trial  condition  i 
and  m  is  the  probit  slope  (equal  to  1/a).  The  50% 
response  level  (or  p  =  0.5)  corresponds  to  a  Z  value 
of  zero.  The  MLE  estimate  of  log(ECT5o)  is  the 
value  of  log(ECT5o)  that  is  found  to  maximize  L  in 
eq  Bl. 
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For  the  MLE  calculations,  the  first  and  second 
derivatives  of  loge(X  ,)  with  respect  to  p  are 
used:3’7’8 
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To  reach  convergence  at  the  value  of  log(ECT50) 
that  maximizes  logeT,  a  Newton-Raphson  (or 
Newton’s  Method)  algorithm  (or  similar 
procedure)  can  be  used.3’7’8’10  Using  the  Newton- 
Raphson  method  for  the  present  system  of 
equations  (eq  Bib,  eq  B6,  and  eq  B7),  the 
following  equation  is  used  to  determine  the  next 
guess  for  log(ECT5o),  as  well  as  to  check  on 
convergence  at  the  MLE  for  log(ECT50): 
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where  p„  is  the  current  guess  for  p  (or  log(ECT50)), 
Pnext  is  the  next  guess  for  p,  and 
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(5)  Multiply  the  L/s  (or  add  the  loge(T;)’s)  together 
to  estimate  the  total  likelihood,  L  (or  logeL),  of  the 
MLE  estimate. 

(6)  Calculate  the  first  and  second  derivatives  for 
logeT  (evaluated  at  p0)  using  eq  B6,  eq  B7,  eq  B9, 
and  eq  BIO. 

(7)  Check  to  verify  whether  the  maximum  value  of 
L  has  been  obtained.  If  not,  go  back  to  Step  (3) 
with  a  new  guess,  pnexl,  for  p  (or  log(ECT50)  using 
eq  B8. 

After  the  final  log(ECT50)  estimate,  ju  ,  is  obtained, 
there  are  three  common  and  general  methods  for 
obtaining  approximate  confidence  limits  for  the 
estimate:3  Wald  test,  likelihood-ratio  test,  and  the 
score  (or  Lagrange-multiplier)  test.  These 
approximations  grow  more  accurate  as  the  sample 
size  gets  larger. 

In  the  present  study,  the  Wald  test  was  used  to 
calculate  confidence  limits.  Limits  from  the  Wald 
test  can  be  readily  obtained  from  calculations 
performed  as  part  of  the  Newton-Raphson 
algorithm  used  for  finding  the  maximum  value  for 
L.  However,  the  likelihood-ratio  test  required 
additional  Newton-Raphson  algorithm  iterations. 

In  the  present  study,  the  following  equation  was 
used  (based  on  the  Wald  test)  to  calculate  the  95% 
asymptotic  confidence  interval  for  p  or  the 
log(ECT50):3 
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The  first  and  second  derivatives  for  logeL  are 
evaluated  at  pQ.  Log J,  is  maximized  when  its  first 
derivative  with  respect  to  p  equals  zero. 
Convergence  is  achieved  when  the  absolute 
difference  between  p0  and  pnext  is  less  than  a 
predetermined  value. 

Thus,  the  following  algorithm  is  used  to  find  the 
MLE  estimate  for  ECT50: 

(1)  Set  the  probit  slope  (m)  equal  to  some  fixed 
value  for  the  duration  of  the  algorithm. 

(2)  Make  an  initial  guess,  p0,  for  p  (or  log(ECT5o)). 

(3)  Calculate  Z, ,  f(Zj)  and  p\  for  each  test  condition 
i,  corresponding  to  some  (CT);  exposure  using  eq 
B2,  eq  B3,  and  eq  B4. 

(4)  Using  eq  Bl,  calculate  the  individual 
likelihoods,  L,. 


(1.96)  „  (1.96) 

-42logcF(A)  -d2  logc  F(p)  (Bl  1) 

d\i2  ]j  d  |X2 

where  the  second  derivatives  for  logLi.  are 
evaluated  at  ju  using  eq  B7  and  eq  BIO. 

B2.2  Example  of  Application  of  Binary 
Probit  Model  with  Fixed  Probit  Slope 

Table  B-l  provides  the  binary  data  for  the  10-min 
exposures  of  the  male  pig  to  GB  vapor  from  Hulet 
et  al.  (2006). 11  Dosage  is  in  units  of  mg-min/m3. 

For  this  example,  test  condition  i  will  only  have 
one  pig.  So,  for  eq  Bl,  n  will  equal  one  for  each 
test  condition.  The  values  for  x,  correspond  to  the 
absence  or  presence  of  lethality  in  the  exposed  pig. 
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Table  B-1.  Male  pig  GB  ordinal  data  (10-min 
exposure  duration) — binary  data 


PIG 

DOSAGE 

(CT) 

logio(CT) 

OUTCOME 

X/ 

1 

53.5 

1.728354 

<  severe 

0 

2 

59.0 

1.770852 

severe 

0 

3 

67.0 

1.826075 

death 

1 

4 

74.5 

1.872156 

severe 

0 

5 

94.0 

1.973128 

death 

1 

6 

95.0 

1.977724 

death 

1 

Steps  (1)  and  (2):  Probit  slope  and  initial  guess 

for  logtn(ECTsn)  for  iteration  one 

For  Step  (1)  of  the  algorithm,  the  probit  slope  is  set 
equal  to  1 0,  which  was  used  as  the  step  size  for  the 
up-and-down  method  employed  in  the  present 
study.  For  Step  (2),  the  initial  guess  for  the 
logio(ECT5o)  is  1.85304. 

Step  (3):  Calculation  of  Z,’s  and  p,  for  iteration 

one  using  eq  B2,  eq  B3,  and  eq  B4 

Z,  =  [1.72835)  -  1.85304)]/(1/10)  =  -1.24685  ==> 

Pi  =0.10623 

Z2  =  [1.77085)  -  1 .85304)]/(  1/10)  =  -0.82187  ==> 
p2  =  0.20558 

Z3  =  [1.82608)  -  1.85304)]/(1/10)  =  -0.26964  ==> 

Pi  =  0.39372 

Z4  =  [1.87216)  -  1.85304)]/(1/10)  =  0.19117  ==> 
p4  =  0.57580 

Z5  =  [1.97313)  -  1.85304)1/(1/10)  =  1.20089  ==> 

Pi  =  0.885 10 

Z6  =  [1.97772)  -  1.85304)]/(1/10)  =  1.24685  ==> 

P& =  0.89377 

Steps  (4)  and  (5):  Calculation  of  IJ_ s  and  L  for 

iteration  one  using  eq  B1 

loge(/,)  =  (0)  (loge(0. 10623))  +  (1  -  0)  (loge(l  - 
0.10623))  =  -0.11230 

loge(Z2)  =  (0)  (loge(0. 20558))  +  (1  -  0)  (loge(l  - 
0.20558))  =  -0.23014 

loge(Z3)  =  (1)  (loge(0. 39372))  +  (1  -  1)  (loge(l  - 
0.39372))  =  -0.93212 


loge(Z4)  =  (0)  (loge(0. 57580))  +  (1  -  0)  (loge(l  - 
0.57580))  =  -0.85756 

loge(Z5)  =  (1)  (loge(0. 88510))  +  (1  -  1)  (loge(l  - 
0.88510))  =  -0.12205 

loge(Z6)  =  (1)  (loge(0. 89377))  +  (1  -  1)  (loge(l  - 
0.89377))  =  -0.11230 

Sum  of  the  above  loge(Zi)’s  (or  loge(Z))  equals 
-2.36647. 

Step  (6):  Calculate  the  first  and  second 

derivatives  for  log,X  (evaluated  at  uJ  using  eq 

B6,  eq  B7,  eq  B9,  and  eq  B10 

d[loge(Z1)]/d[p0]  =  2.05162 

d[loge(Z2)]/d[p0]  =  3.58246 

d[loge(Z3)]/d[p0]  =  -9.77094 

d[loge(Z4)]/d[p0]  =  9.23437 

d[loge(Z5)]/d[p0]  =  -2.19159 

d[loge(Z6)]/d[p0]  =  -2.05162 

Sum  of  the  above  d[loge(Zi)]/d[p0]’s  equals 
0.85430. 

d2[loge(Z1)]/d[p0]2  =  -29.790 

d2[loge(Z2)]/d[p0]2  =  -42.277 

d2[loge(Z3)]/d[|i0]2  =  -69.125 

d2[loge(Z4)]/d[p0]2  =  -67.620 

d2[loge(Z5)]/d[|i0]2  =  -31.122 

d2[loge(Z6)]/d[|i0]2  =  -29.790 

Sum  of  the  above  d2[loge(Zi)]/d[p0]2  ’s  equals 
-269.72. 

Step  (7):  Check  for  convergence  on  maximum  L 

value  and  new  guess  for  log(ECTSn)  for  iteration 
two  using  eq  B8 

After  the  first  iteration,  the  next  guess  for  p  is 
found  to  equal: 

pnext  =  1.85304  -  (0.85430)/(-269.72)  =  1.85304  - 
(-0.00317)  =  1.85621 

Convergence  was  nearly  reached  after  the  first 
iteration,  as  seen  above  with  only  a  difference  of 
-0.00317  between  pnext  and  p„.  After  the  second 
iteration,  the  difference  falls  further  to  2.1  x  10'7. 
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Thus,  the  final  estimate  for  logio(ECT50)  is  1.85621 
(or  (ECT50)  =  71.8  mg-min/m3),  and  the  final 
loge(E)  value  was  -2.36512.  The  denominators  of 
eq  Bll  were  found  to  equal  the  square  root  of 
269.7.  With  this  value  for  the  denominators,  the 
corresponding  95%  asymptotic  confidence  interval 
for  logio(ECT50)  equals  1.73686  to  1.97556,  or  for 
ECT50,  the  interval  is  54.6  to  94.5  mg-min/m3. 

B3.0  Ordinal  Probit  Model 

B3.1  MLE  Algorithm  for  Ordinal  Probit 
Model 

To  model  an  ordinal  ternary  response,  eq  B1  is 
modified  as  follows  for  each  trial  condition  z: 


T  *1./  x2  i  /  A  \ni  - 

Li  =  Pv  P2,i  l1  -  Pu~  P2,i) 


(B12a) 
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m  {log-io (CT)i  -  log10  (ECTS0 )}  (B15) 
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m  {loQio  (CT),  -  log10(LCT50)}  (B16) 


where  P2  >  Pi  and  a  (or  m )  is  assumed  to  be  the 
same  for  both  distributions.  MLE  is  now  used  to 
simultaneously  obtain  estimates  for  both  the  ECT50 
(severe)  and  LCT50.  However,  some  modifications 
are  needed  to  eq  B6  and  eq  B7  to  account  for  the 
first  and  second  derivatives  of  loge(T,)  with  respect 
to  pi  and  P2,  respectively: 
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where  p  14  and  p2, i  are  the  event  probabilities  for 
responses  of  categories  one  and  two,  respectively, 
for  test  condition  i,  tit  is  the  number  of  independent 
trials  under  the  z'-th  condition,  and  xtl  and  x2j  are 
the  number  of  responses  of  categories  one  and  two, 
respectively,  in  n,.  As  before  with  the  binary 
model,  the  likelihood,  L ,  is  obtained  by  multiplying 
together  the  individual  likelihoods  (for  all  test 
conditions). 

The  ordinal  categories  are  in  order  of  increasing 
severity,  with  Category  Two  being  more  severe 
than  Category  One.  For  this  example,  Category 
Two  corresponds  to  lethality,  while  Category  One 
corresponds  to  severe  effects.  Two  normal 
distributions  are  represented  in  eq  12,  and  are 
defined  as  follows  for  the  z'th  condition: 


P2.,=  f  2"  f{Z)dZ 
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P\i+P2,i=  f  f{z)dz 
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(B14) 

The  values  of  the  individual  p\/s  and  p2fs  are  a 
function  of  the  applied  dosages,  (C7)„  used  in  an 
experiment  and  their  respective  distances  from 
their  corresponding  median  effective  dosages:  for 
category  one,  pi  or  ECT50  (severe);  and  for 
category  two,  p2  or  LCT50  (lethality).  This  is 
reflected  in  the  following  definitions  of  the  Z,’s  for 
each  z'-th  condition: 
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where  po. ,  equals  (1  -  pi ,  -  p2. ,).  Also,  the  partial 
derivative  of  loge(A  ,)  with  respect  to  both  p!  and 
P2,  is  required: 
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To  reach  convergence  at  the  values  for  pj  (or 
ECT50  (severe))  and  p2  (or  LCT50  (lethality))  that 
maximizes  logeT,  a  Newton-Raphson  (or  Newton’s 
Method)  algorithm  (or  similar  procedure)  can  be 
used.3'7'8'10  Using  the  Newton-Raphson  method  for 
the  present  system  of  equations  (eq  B17  to  eq 
B21),  the  following  simultaneous  equations  are 
used  to  determine  the  next  guess  for  the  median 
effective  dosages,  as  well  as  to  check  on  the 
convergence  of  the  solution: 
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where  Apj  =  (|incxt  -  J-L0)i  and  Ap2  =  (pnext  -  p0)2  for 
median  effective  dosages  1  (severe)  and  2  (lethal), 
respectively,  and 
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where,  j  equals  1  and  2  for  severe  and  lethality, 
respectively.  The  above  derivatives  for  logeT  are 
evaluated  at  pij0  and  p2,0.  LogeT  is  maximized 
when  its  first  derivatives  with  respect  to  the  two 
p’s  equal  zero.  Convergence  is  achieved  when  the 
absolute  difference  between  p0  and  pnext  is  less  than 
a  predetermined  value.  After  convergence  is 
reached,  the  asymptotic  variance-covariance 
matrix  of  the  maximum  likelihood  estimate  can  be 
calculated  by  taking  the  inverse  of  the  matrix  of 
second  derivatives  of  logek:3’7’8 
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Thus,  the  following  algorithm  is  used  to  find  the 

MLE  estimate  for  ECT50: 

(1)  Set  the  probit  slope  ( m )  equal  to  some  fixed 
value  for  the  duration  of  the  algorithm. 

(2)  Make  initial  guesses  for  log(ECTso)  and 
log(LCT50):  pi,0  and  p2,0,  respectively. 

(3)  Calculate  Z,j  ,  f(Zuj)  and  p  y  for  each  test 
condition  i  and  mean  dosage  j,  corresponding 
to  some  (CT)i  using  eq  B3  and  eq  B13  to  eq 
B16. 

(4)  Using  eq  B12,  calculate  the  individual 
likelihoods,  L,. 

(5)  Multiply  the  L' s  (or  add  the  loge(T/)’s) 
together  to  estimate  the  total  likelihood,  L  (or 
logek),  of  the  MLE  estimate. 

(6)  Calculate  the  first  and  second  derivatives  for 
logeT  (evaluated  at  p0)  for  each  mean  dosage  j, 
using  eq  B17  to  eq  B20.  Also,  calculate  the 


derivative  for  logeZ.  with  respect  to  both  p ,  0 
and  p2  o  using  eq  B2 1 . 

(7)  Check  to  verify  whether  the  maximum  value  of 
L  has  been  obtained.  If  not,  go  back  to  Step  (3) 
with  new  guesses,  p^next  and  p2,next,  for  pi  and 
p2,  respectively,  by  solving  eq  B22  and  eq 
B23,  simultaneously. 

In  some  instances  with  Step  (7),  poor  initial 
guesses  for  pi  and  p2  may  produce  the  situation 
where  pi,next  >  P2,next,  a  violation  of  a  key  boundary 
condition.  A  simple  resolution  for  this  problem  is 
to  artificially  reduce  (for  this  iteration)  the  values 
of  Ap!  and  Ap2  by  some  set  factor. 

After  the  final  log(ECTso)  and  log(LCT50) 
estimates,  jjl  and  ju2  ,  are  obtained,  there  are  three 

common  and  general  methods  for  obtaining 
approximate  confidence  limits  for  these  estimates:3 
Wald  test,  likelihood-ratio  test,  and  the  score  (or 
Lagrange-multiplier)  test.  As  with  the  binary  probit 
model  (see  Section  B2.0),  these  approximations 
grow  more  accurate  as  the  sample  size  gets  larger. 

In  the  present  study,  the  Wald  test  was  used  to 
calculate  confidence  limits.  Limits  from  the  Wald 
test  can  be  readily  obtained  from  calculations 
performed  as  part  of  the  Newton-Raphson 
algorithm  used  for  finding  the  maximum  value  for 
L.  However,  the  likelihood-ratio  test  required 
additional  Newton-Raphson  algorithm  iterations. 

In  the  present  study,  the  following  equation  was 
used  (based  on  the  Wald  test)  to  calculate  the  95% 
asymptotic  confidence  interval  for  each  pj  [with  j  = 
1  for  log(ECT50)  and  2  for  log(LCT50)]:3’7’8 
where  vatf/p)  is  the  variance  for  ft,.  The  values  for 
the  variances  and  covariance  are  calculated  using 
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eq  B27. 

B3.2  Example  of  Application  of  Ordinal 
Probit  Model  with  Fixed  Probit  Slope 

Table  B-2  provides  the  quantal  data  for  the  10-min 
exposures  of  the  male  pig  to  GB  vapor  from  Hulet 
et  al.  (2006). 11  Dosage  is  in  units  of  mg-min/nT. 
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For  this  example,  test  condition  i  will  only  have 
one  pig.  So,  n  will  equal  one  for  each  test  condition 
for  eq  B12.  The  values  for  xj  and  x2  correspond  to 
the  absence  or  presence  of  a  maximum  effect:  xj 
equals  one  if  the  maximum  effect  observed  was 
severe  effects  (and  equals  zero  otherwise);  and  x2 
equals  one  if  the  maximum  effect  was  lethality 
(and  equals  zero  otherwise). 

Steps  (1)  and  (2):  Probit  slope  and  initial 

guesses  for  logm(ECTsn)  and  logm(LCTsn)  for 
iteration  one. 

For  Step  (1)  of  the  algorithm,  the  probit  slope  is  set 
equal  to  1 0,  which  was  used  as  the  step  size  for  the 
up  and  down  method  employed  in  the  present 
study.  For  Step  (2),  the  initial  guesses  for 
logio(ECT50)  and  logi0(LCT50)  are  1.68  and  1.83, 
respectively. 

Steps  (3),  (4)  and  (51:  Calculation  of  Z,  r  ,  f(Z: ,) 
and  p  /  ,  using  eq  B3  and  eq  B13  to  eq  B16  and  of 

Z,,’s  and  L  Using  eq  B12  for  iteration  one. 

Table  B-3  shows  the  calculated  values  for  each 
pig.  The  sum  of  the  individual  logeT,  equals 
-3.8212. 

Step  (6):  Calculate  the  various  derivatives  for 

log,X  (evaluated  at  m  n  and  ni  „)  for  each  mean 

dosage  /,  using  eq  B17  to  eq  B21. 

Table  B-4  shows  the  calculated  values  for  each 
pig.  Pigs  showing  severe  effects  make 
contributions  to  all  five  derivatives,  whereas  those 
pigs  having  less  than  severe  effects  or  that  have 
died  only  contribute  to  two  of  the  five  derivatives. 
Step  (7):  Check  for  convergence  on  maximum  L 

value  and  new  guesses  for  log(ECTsn)  and 
log(LCTsn)  for  iteration  two  using  eq  B22  and 

eq  B23. 

Solving  eq  B22  and  eq  B23  simultaneously 


Table  B-2.  Male  pig  GB  ordinal  data  (10-min 
exposure  duration) — quantal  data 


PIG 

DOSAGE 

(CT) 

logio(CT) 

OUTCOME 

Xl 

X2 

1 

53.5 

1.728354 

<  severe 

0 

0 

2 

59.0 

1.770852 

severe 

1 

0 

3 

67.0 

1.826075 

death 

0 

1 

4 

74.5 

1.872156 

severe 

1 

0 

5 

94.0 

1.973128 

death 

0 

1 

6 

95.0 

1.977724 

death 

0 

1 

produces  values  of  0.03283  and  0.03087  for  Api 
and  Ap2,  respectively.  So,  the  next  guesses  for  pi 
and  p2  are  1.71283  and  1.86087,  respectively. 
After  the  third  iteration,  the  absolute  values  for 
both  Api  and  Ap2  are  less  than  1  x  10°.  At  which 
point,  the  final  values  for  pi  and  p2  are  1.71203 
and  1.86015,  respectively.  Thus,  the  final  estimates 

for  ECT5o  and  LCT50  are  51.5  and  72.5  mg-min/m3, 
respectively.  The  final  loge(T)  value  was  -3.6478. 
The  final  variance-covariance  matrix  equals: 


0.005449  0.001100 

0.001100  0.003642 


Using  the  above  information  with  eq  B28,  the 
corresponding  95%  asymptotic  confidence  interval 
for  logio(ECT50)  equals  1.56735  to  1.85671,  or  for 
ECT50,  the  interval  is  36.9  to  71.9  mg-min/m3.  For 
lethality,  the  confidence  interval  for  log10(LCT50) 
equals  1.74186  to  1.97845,  or  for  LCT50,  the 
interval  is  55.2  to  95.2  mg-min/m3. 


Table  B-3.  Values  of  Zy,  f(Zi:j)  and  p,y  for  first  iteration 


PIG 

Zi 

Z2 

m 

m 

Pi 

P2 

loge  L 

1 

0.48354 

-1.01646 

0.35493 

0.23799 

0.53094 

0.15470 

-1.15720 

2 

0.90852 

-0.59148 

0.26404 

0.33492 

0.54110 

0.27710 

-0.61420 

3 

1.46075 

-0.03925 

0.13727 

0.39864 

0.44361 

0.48434 

-0.72500 

4 

1.92156 

0.42156 

0.06297 

0.36502 

0.30934 

0.66333 

-1.17330 

5 

2.93128 

1.43128 

0.00543 

0.14324 

0.07449 

0.92382 

-0.07920 

6 

2.97724 

1.47724 

0.00474 

0.13398 

0.06835 

0.93019 

-0.07240 
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Table  B-4.  Values  of  logeL  derivatives  for  first  iteration 


1 

11.2906 

-72.8829 

0 

0 

0 

2 

-4.8798 

-68.1456 

6.1896 

-74.9220 

30.2039 

3 

0 

0 

-8.2304 

-64.5089 

0 

4 

-2.0355 

-43.2572 

11.8000 

-89.4952 

24.0191 

5 

0 

0 

-1.5505 

-24.5966 

0 

6 

0 

0 

-1.4404 

-23.3522 

0 

Total 

4.3753 

-184.2857 

6.7683 

-276.8749 

54.2231 

B4.0  Other  Useful  Relationships 

For  simplicity,  many  of  the  derivative  equations 
are  presented  without  benefit  of  showing 
intermediate  steps.  The  following  is  a  listing  of 
relationships  that  were  useful  in  arriving  at  the 
final  relations. 

B4.1  Binary  Probit  Model 


dp,  _ 

dp 

y(z) 

dp 


-m) 


f(z)z 


B4.2  Ordinal  Probit  Model 


5Z, 

dp 


(B29) 


(B30) 


(B31) 


foi,. 

dp i 

dp  2., 

dp, 

dp,,, 

dju  2 


-f(zP 


=  0 


r  ^  'n 

dp  2., 
dju  2 


/(Z2„) 


(B32) 


(B33) 


(B34) 


dp 2, _  -f(Z2,i) 


dp2 

df(Z ,) 
dp i 

dp  2 

df{Z2 ) 


f(Z2)Z2 


df  (Zf ) 


dpy  dp2 

5Z1  dZ2 
dp i  dp 2 

8Z.  dZ2 
— -  =  — =  ( 
dp2  dp, 


=  0 


'  1 

ler 


(B35) 


(B36) 


(B37) 


(B38) 


(B39) 


(B40) 
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APPENDIX  C:  MAXIMUM  LIKELIHOOD  ESTIMATION  CALCULATIONS  OF  LCT50  VALUES 
FOR  DATA  OF  LEHMANN  (1882)  AND  McELROY  ET  AL.  (1943)— Douglas  R.  Sommerville 


C1.0  Introduction 

The  literature  search  for  this  study  discovered  two 
datasets  for  which  LCT50  values  were  not 
originally  reported:  Lehmann  (1882)1  and  McElroy 
et  al.  (1943). 2  In  both  cases,  the  data  are 
insufficient  for  traditional  probit  analysis  to 
provide  a  LCT50  estimate.  However,  if  a  one- 
parameter  MLE  analysis  is  used  instead  of  the  two- 
parameter  traditional  probit  analysis,  it  is  possible 
to  obtain  LCT50  values  from  these  datasets.  The 
data  from  these  studies  and  the  resulting  MLE 


calculations  and  LCT50  estimates  are  presented  in 
this  appendix. 

C2.0  Experimental  Data 

The  experimental  data  of  Lehmann  [(1882)1  (as 
reported  by  the  Major  Hazards  Assessment  Panel 
(1987)]3  and  McElroy  et  al.  (1943)2  are  listed  in 
Tables  C-l  through  C-4.  The  data  from  Lehmann 
are  also  shown  in  Figures  C-l  through  C-3  (pages 
C-3  and  C-4). 


Table  C-1.  Individual  response  data  (72-h  basis)  for  cats  exposed  to  chlorine  in  Lehmann 
(1882) 


CONCENTRATION 

TIME 

CT 

C2T 

m  iTrnMF 

ppm 

mg/m3 

(min) 

(mg-min/m3) 

(mg2-min/m6) 

10 

28.98 

450 

13000 

3.78E+05 

survived 

130 

377 

630 

237000 

8.94E+07 

survived 

180 

522 

300 

156000 

8.16E+07 

survived 

330 

956 

390 

373000 

3.57E+08 

survived 

450 

1300 

210 

274000 

3.57E+08 

died 

430 

1250 

300 

374000 

4.66E+08 

survived 

660 

1910 

240 

459000 

8.78E+08 

survived 

3100 

8980 

180 

1617000 

1.45E+10 

died 

3300 

9560 

90 

861000 

8.23E+09 

died 

4500 

13000 

60 

782000 

1.02E+10 

died 

Table  C-2.  Individual  response  data  (72-h  basis)  for  rabbits  exposed  to  chlorine  in  Lehmann 


(1882) 


CONCENTRATION 

TIME 

CT 

C2T 

m  iTrnMc 

ppm 

mg/m3 

(min) 

(mg-min/m3) 

(mg2-min/m6) 

180 

522 

300 

156000 

8.16E+07 

survived 

330 

956 

390 

373000 

3.57E+08 

survived 

400 

1160 

210 

243000 

2.82E+08 

died 

430 

1250 

300 

374000 

4.66E+08 

died 

660 

1910 

240 

459000 

8.78E+08 

survived 

3100 

8980 

180 

1617000 

1.45E+10 

died 

3300 

9560 

90 

861000 

8.23E+09 

died 

4500 

13000 

60 

782000 

1.02E+10 

died 
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Table  C-3.  Individual  response  data  (72-h  basis)  for  guinea  pigs  exposed  to  chlorine  in 
Lehmann  (1882) 


CONCENTRATION 

TIME 

CT 

C2T 

m  iTrnMF 

ppm 

mg/m3 

(min) 

(mg-min/m3) 

(mg2-min/m6) 

180 

522 

300 

156000 

8.16E+07 

survived 

330 

956 

390 

373000 

3.57E+08 

died 

450 

1300 

210 

274000 

3.57E+08 

survived 

430 

1250 

300 

374000 

4.66E+08 

survived 

660 

1910 

240 

459000 

8.78E+08 

died 

3200 

9270 

180 

1670000 

1.55E+10 

died 

3300 

9560 

60 

574000 

5.49E+09 

died 

4500 

13000 

60 

782000 

1.02E+10 

died 

Table  C-4.  Individual  response  data  (72-h  basis)  for  goats  exposed  to  chlorine  in  McElroy  et  al. 


(1943) 


CONCENTRATION 

TIME 

CT 

C2T 

ni  rrrniwip 

mg/m3 

(min) 

(mg-min/m3) 

(mg2-min/m6) 

3000 

20 

60000 

1.80E+08 

died 

3000 

20 

60000 

1.80E+08 

survived 

2500 

17.5 

43750 

1.09E+08 

died 

2500 

17.5 

43750 

1.09E+08 

survived 

2500 

17.5 

43750 

1.09E+08 

survived 

2600 

10 

26000 

6.76E+07 

died 

1900 

8 

15200 

2.89E+07 

died 

1900 

8 

15200 

2.89E+07 

survived 

1900 

8 

15200 

2.89E+07 

survived 

C3.0  Data  Reduction  and  Analysis 

The  data  from  Lehmann  (1882)  were  grouped  by 
species,  and  a  MLE  analysis  (see  Appendix  A)  was 
applied  separately  to  each.  The  probit  slope  was 
assumed  to  equal  10.  It  was  assumed  based  on  the 
total  historical  dataset  that  chlorine  toxicity  follows 
a  toxic  load  type  model  for  time  dependence.  With 
this  in  mind,  the  binary  response  was  analyzed 
against  the  logarithm  (base  1 0)  of  two  forms  of  the 
toxic  load:  C2T  and  C3T.  It  was  found  that  the  final 
estimate  was  relatively  insensitive  to  the  value  of 
the  toxic  load  exponent  being  used  (two  versus 
three).  Thus,  for  the  final  estimate,  C2T  was  used 
for  the  toxic  load  model.  After  the  median  lethal 
C2T  was  determined,  the  geometric  mean  of  all  the 
individual  exposure  durations  (T)  was  then 
calculated  for  each  species  dataset.  The  L(C2T)50 
was  assumed  to  be  associated  with  this  geometric 


mean  time,  which  was  found  to  equal  235  min  for 
the  cat,  190  min  for  the  rabbit  and  180  min  for  the 
guinea  pig.  The  LC5o  and  LCT50  was  then 
calculated  from  the  L(C2T)50  values  using  these 
mean  times. 

The  data  from  McElroy  et  al.  (1943)2  was  split  into 
two  exposures  groups:  short-time  durations  (8  min 
and  10  min)  and  long-time  durations  (17.5  min  and 
20  min).  MLE  analysis  was  then  used  on  each  set 
separately.  The  geometric  mean  of  all  of  the 
individual  exposure  durations  (T)  within  each  set 
was  taken.  As  with  the  Lehmann  datasets,  L(C2T)50 
values  were  assumed  to  be  associated  with  the 
appropriate  geometric  mean  times.  For  the  short- 
time  duration,  the  geometric  mean  is  8.5  min,  and 
for  the  long  duration,  it  was  18.5  min.  The  LC50 
and  LCT50  was  then  calculated  from  the  L(C2T)50 
values  using  these  mean  times. 
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Figure  C-1.  Binary  response  data  for  chlorine  inhalation  exposures  for  cats  from 
Lehmann  (1882) 
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Figure  C-2.  Binary  response  data  for  chlorine  inhalation  exposures  for  rabbits 
from  Lehmann  (1882) 
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Figure  C-3.  Binary  response  data  for  chlorine  inhalation  exposures  for  guinea 
pigs  from  Lehmann  (1882) 


C4.0  Discussion  and  Results 

The  estimated  LCT50  values  for  both  Lehmann 
(1882)1  and  McElroy  et  al.  (1943)2  are  listed  in 
Table  C-5  (with  approximate  95%  confidence 
intervals),  and  the  values  for  the  Lehmann  dataset 
are  shown  in  Figures  C-l  to  C-3. 

Based  upon  the  ratio  of  the  95%  confidence  limits, 
the  LCT5o  estimates  for  McElroy  et  al.  dataset  are 
more  precise  than  those  for  Lehmann.  This  is 
probably  due  to  McElroy  et  al.  having  more 
animals  exposed  at  some  fixed  duration  (or  small 
duration  interval)  than  Lehmann.  In  the  case  of 
Lehmann,  it  was  infrequent  for  him  to  have  more 
than  one  exposure  for  a  particular  duration. 

For  best  results,  a  one -parameter  MLE  analysis 
should  be  used  with  one-dimensional  datasets.  This 
is  usually  the  case  with  response  data  collected  for 
a  series  of  different  vapor  concentrations  at  one 
exposure  duration.  With  McElroy  et  al.  (1943), 2 
this  was  readily  accomplished  (see  above)  through 
judicious  grouping.  For  Lehmann  (1882),1  this 
could  not  be  done,  and  at  first  glance,  the  data 
appear  to  be  two-dimensional,  with  both  concen¬ 
tration  and  duration  being  varied  simultaneously. 
However,  the  coordinates  for  the  test  conditions  do 
roughly  stay  within  a  narrow  band  and  thus  are 
one-dimensional.  This  band  is  more  or  less 


perpendicular  to  the  toxic  load  model  extrapolation 
from  the  estimated  LCT50  value.  This  is 
demonstrated  in  Figure  C-l  using  Lehmann’s  cat 
data.  Two  different  toxic  load  extrapolations  are 
shown:  n  =2  and  n  =  3.  With  a  proper  MLE 
application,  the  estimated  median  effective 
dosage/toxic  load  should  be  a  demarcation  point 
between  two  groups:  one  predominated  by  lethal 
outcomes  (above  the  median  effective  toxic  load) 
and  the  other  by  nonlethal  outcomes  (below  the 
median  effective  toxic  load).  For  the  Lehmann 
data,  such  an  effect  is  seen.  Also,  there  is  no 
difference  in  how  the  L(C2T)50  and  L(C3T)50  curves 
split  the  test  coordinates,  which  is  why  the  MLE 
estimate  for  the  median  effective  dosage  is 
insensitive  (in  this  case)  to  the  assumed  value  for 
the  toxic  load  exponent  (n). 

Prior  to  a  MLE  analysis  approach,  it  was  not 
possible  to  derive  estimated  LCT50  values  for  either 
Lehmann  or  McElroy  et  al.  using  traditional  probit 
analysis  methods.  Thus,  MLE  analysis  can 
“retrieve”  additional  median  lethal  toxicity 
information  from  previously  ignored  data  sources. 
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Table  C-5.  LCT50  estimates  from  MLE  analysis  of  datasets  from  Lehmann  (1882)  and  McElroy  et 


al.  (1943) 


SPECIES 

TIME 

(min) 

LCTso 

(mg-min/m3) 

APPROXIMATE  95% 
CONFIDENCE  INTERVAL 

RATIO  OF 

i  ii  t n  1 1 

STUDY 

LOWER  LIMIT 

UPPER  LIMIT 

Goat 

8.5 

16500 

13900 

19500 

1.40 

McElroy  et  al. 

Goat 

18.5 

51500 

44500 

59000 

1.33 

McElroy  et  al. 

Cat 

235 

390000 

290000 

520000 

1.79 

Lehmann 

Rabbit 

190 

300000 

225000 

385000 

1.71 

Lehmann 

Guinea  Pig 

180 

290000 

210000 

390000 

1.86 

Lehmann 
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APPENDIX  D:  COMPLETE  TABLE  OF  DATA  AND  COMPUTATIONS 


Table  D-1.  Table  of  data  and  calculations 


STUDY  (AUTHORS) 

REPORT 

DATE 

REFERENCE 

Nl  1IVFSFR 

RAW 

DATA 

IVETHOD 

TIIVE 

LCTso 

(mg- 

PROBIT 

SLOPE 

SLOPE 

ERROR 

MV 

(\) 

MV 

1  rrv>1 

MASS 

tko) 

LETHAL 

DOSE 

LOG 
(\  n) 

LOG(M)  LOG(T) 

min/m?) 

Mouse 

Slver,  McGrath 

1942 

i 

Yes 

Probit 

10 

13150 

7.3207 

0.5184 

0.031 

0.000031 

0.0207 

0.40765 

-0.3897 

-1.6840 

1.0000 

Slver,  McGrath,  Ferguson 

1942 

2 

Yes 

Probit 

10 

19430 

7.8260 

1.0520 

0.031 

0.000031 

0.0207 

0.60233 

-0.2202 

-1.6840 

1.0000 

Zwart,  Wcutersen 

1988 

3 

Yes 

Probit 

10 

30620 

10.0410 

2.0060 

0.031 

0.000031 

0.0207 

0.94922 

-0.0226 

-1.6840 

1.0000 

Alarie 

1980 

4 

No 

10 

8752 

0.031 

0.000031 

0.0207 

0.27131 

-0.5665 

-1.6840 

1.0000 

Bunting 

1945 

5 

No 

10 

15200 

0.031 

0.000031 

0.0207 

0.47120 

-0.3268 

-1.6840 

1.0000 

Upton,  Rotariu 

1941 

6 

No 

10 

18199 

0.031 

0.000031 

0.0207 

0.56417 

-0.2486 

-1.6840 

1.0000 

Btron,  Aharonson 

1978 

7 

Yes 

IVLE 

11 

9245 

10.6300 

1.0160 

0.031 

0.000031 

0.0207 

0.28660 

-0.5427 

-1.6840 

1.0414 

Schlagbauer,  Flenschler 

1967 

8 

Yes 

Probit 

30 

10682 

6.1790 

1.2280 

0.031 

0.000031 

0.0207 

0.33114 

-0.4800 

-1.6840 

1.4771 

Zwart,  Wcutersen 

1988 

3 

Yes 

Probit 

30 

49300 

6.8240 

3.6750 

0.031 

0.000031 

0.0207 

1.52830 

0.1842 

-1.6840 

1.4771 

Btron,  Aharonson 

1978 

7 

Yes 

IVLE 

55 

27096 

10.6300 

1.0160 

0.031 

0.000031 

0.0207 

0.83998 

-0.0757 

-1.6840 

1.7404 

Back,  Thomas,  MacEwen 

1972 

9 

No 

60 

23822 

0.031 

0.000031 

0.0207 

0.73848 

-0.1317 

-1.6840 

1.7782 

Rat 

Zwart,  Wcutersen 

1988 

3 

Yes 

Probit 

5 

77850 

15.8410 

3.3290 

0.203 

0.000203 

0.2610 

15.80355 

1.1988 

-0.5834 

0.6990 

Zwart,  Wcutersen 

1988 

3 

Yes 

Probit 

10 

61160 

15.8410 

3.3290 

0.203 

0.000203 

0.2610 

12.41548 

1.0940 

-0.5834 

1.0000 

Zwart,  Wcutersen 

1988 

3 

Yes 

Probit 

30 

54810 

15.8410 

3.3290 

0.203 

0.000203 

0.2610 

11.12643 

1.0464 

-0.5834 

1.4771 

Zwart,  Wcutersen 

1988 

3 

Yes 

Probit 

60 

83820 

15.8410 

3.3290 

0.203 

0.000203 

0.2610 

17.01546 

1.2308 

-0.5834 

1.7782 

Vemot,  MacEwen,  Haun 

1977 

10 

No 

60 

50947 

0.203 

0.000203 

0.2610 

10.34224 

1.0146 

-0.5834 

1.7782 

Guinea  Rg 

Lehmann 

1882 

11 

Yes 

IVLE 

180 

290000 

0.182 

0.000182 

0.3230 

52.78000 

1.7225 

-0.4908 

2.2553 
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Table  D-1.  Table  of  data  and  calculations 


STUDY  (AUTHORS) 

REPORT 

DATE 

REFERENCE 

NUIVBER 

RAW 

DATA 

IVETHOD 

T1IVE 

(min) 

LCTso 

(mo- 
mi  n/m?) 

PROBIT 

SLOPE 

SLOPE 

ERROR 

MV 

(L) 

MV 

(mP) 

MASS 

(kg) 

LETHAL 

DOSE 

(mg) 

LOG 

(LD) 

LOG(M)  LOG(T) 

Rabbit 

Marshall 

1917(a) 

12 

Yes 

IVLE 

30 

21580 

1.453 

0.001453 

2.7700 

31.35574 

1.4963 

0.4425 

1.4771 

Barrow,  Smith 

1975 

13 

Yes 

IVLE 

30 

30479 

1.453 

0.001453 

2.7700 

44.28599 

1.6463 

0.4425 

1.4771 

Lehmann 

1882 

11 

Yes 

IVLE 

190 

300000 

1.453 

0.001453 

2.7700 

435.90000 

2.6394 

0.4425 

2.2788 

Cat 

Lehmann 

1882 

11 

Yes 

IVLE 

235 

390000 

0.845 

0.000845 

3.3600 

329.55000 

2.5179 

0.5263 

2.3711 

Dog 

Armstrong 

1923 

14 

Yes 

Probit 

10 

58340 

7.5120 

2.8290 

4.230 

0.004230 

13.0000 

246.77820 

2.3923 

1.1139 

1.0000 

Marshall 

1917(a) 

12 

Yes 

Probit 

30 

47010 

3.1170 

1.4460 

4.230 

0.004230 

13.0000 

198.85230 

2.2985 

1.1139 

1.4771 

Underhill 

1920 

15 

Yes 

Probit 

30 

49380 

3.4514 

0.8053 

4.230 

0.004230 

13.0000 

208.87740 

2.3199 

1.1139 

1.4771 

Marshall 

1917(b) 

16 

No 

30 

75000 

4.230 

0.004230 

13.0000 

317.25000 

2.5014 

1.1139 

1.4771 

Beebe 

1924 

17 

No 

30 

90000 

4.230 

0.004230 

13.0000 

380.70000 

2.5806 

1.1139 

1.4771 

Sheep 

Batchinsky,  Martini,  Jordan 

2006 

18 

No 

30 

24343 

13.900 

0.013900 

52.6000 

338.36770 

2.5294 

1.7210 

1.4771 

Goat 

McElroy,  Shils,  Qnsburg 

1943 

19 

Yes 

IVLE 

18.5 

51500 

9.600 

0.009600 

36.9000 

494.40000 

2.6941 

1.5670 

1.2672 

McBroy,  Shils,  Qnsburg 

1943 

19 

Yes 

IVLE 

8.5 

16500 

9.600 

0.009600 

36.9000 

158.40000 

2.1998 

1.5670 

0.9294 

FOUO 
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APPENDIX  E:  ITERATIVELY  REWEIGHTED  LEAST  SQUARES  (IRLS) 

i  =  1  (Establish  iteration  counter,  where  i  =  1,  2,  .  . 

.m,  and  “m”  is  the  total  number  of  iterations 
needed  for  convergence. 

i 

i 

Derive  first  (i  =  1)  regression  equation.  The  initial 
regression  is  unweighted,  since  the  weights  are  a 
function  of  the  studentized  residuals  that  will  occur 
after  the  first  model  is  attempted.  The  residuals 
from  any  iteration  will  generate  the  weights  for 
producing  the  subsequent  iteration  (if  one  is 
needed).  The  form  of  the  model  is: 

log(Dose)  =  ka  +  k,„  log(Mass)  +  k,  log(Time) 

I 

I 

Calculate  the  studentized  residual  for  each 
observed  value.  The  residual  for  the  ith  observation 
is  calculated  by  a  comparison  of  the  full  model 
with  a  model  with  the  ith  observation  removed. 

These  quantities  are  labeled  as  TRESij  where: 

j  =  1 ,  2,  .  .  .  .  n,  and  “n”  is  the  number  of 
observations. 

I 

I 

Calculate  the  set  of  “weights”  to  be  applied  to  the 
observations  in  the  second  iteration. 

W2j  =  2/(1  +  [TRESij]2)  where  j  =  1,  2,  ....  n, 
and  “n”  is  the  number  of  observations. 

I 

I 

Derive  second  (i  =  2)  regression  equation. 

I 

I 

Calculate  t-values  for  the  second  set  of 
standardized  residuals  for  use  in  the  third  iteration 
([TRES2j]2) 

I 

I 

Continued  on  next  page. 
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Continued  from  previous  page. 
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I 

I 

— »  i  =  i  +  1 

I 

I 


Calculate  weights  for  the  “  ith  ”  iteration. 

I 

I 

Wy  =  2/(1  +  [TRESj.ij]2).3 

I 

I 

Derive  “  ith  ”  regression  equation. 

I 

i 

Calculate  the  studentized  residuals  ([TRESy]2). 

i 

i 

Calculate  SUM  =  X  (Wy  —  Wmj)2  (sum  over  “n”  values). 

i 

i 

Is  this  SUM  less  than  some  tolerance  level  (Such  as  .001  or  .0001)? 

NO  YES 

|  I 

|  I 

i  =  i  +  1  -l 

i  i 

i  i 

i 

Convergence  (within  tolerance  level)  has  been  achieved. 

i 

i 


Declare  the  final  regression  model. 


aA  supplemental  approach  that  was  used  in  the  weighted  regression  runs  for  this  project  incoiporates 
an  averaging  process.  Weights  are  calculated  as  a  result  of  the  studentized  residuals,  as  presented 
above.  These  values  can  be  referred  to  as  “preliminary”  weights.  This  new  set  of  weights  is  averaged 
with  the  weights  used  in  the  run  immediately  prior  to  form  the  “applied”  weights  for  the  current 
model.  That  is, 

W)j  (applied)  [  ^Y.j  (preliminary)  ^  W , .  |  j  (applied)  ]/2 

This  process,  though  optional,  may  induce  an  earlier  convergence  and  a  quicker  adherence  to  the 
stated  tolerance  level. 
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APPENDIX  F:  MINITAB®  SESSION  IRLS  ANALYSIS  FOR  LOG(LD50)  AS  A  FUNCTION  OF 
LOG(TIME)  AND  LOG(MASS) 


Regression  analysis:  log(LDso)  versus  log(M), 
log(T) 

Unweighted  regression: 

The  regression  equation  is: 

Log(LDso)  =  0.642  +  0.873  log(M)  +  0.543  log(T) 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

0.6420 

0.1874 

3.43 

0.002 

Log(M) 

0.87255 

0.04281 

20.38 

0.000 

1.0 

Log(T) 

0.5433 

0.1259 

4.32 

0.000 

1.0 

S  =  0.2812 
R-Sq  =  94.9% 
R-Sq(adj)  =  94.5% 
PRESS  =  2.63845 
R-Sq(pred)  =  93.50% 


Analysis  of  variance 


Source 

DF 

SS 

MS  F 

P 

Regression 

2 

38.530 

19.265  243.63 

0.000 

Residual  error 

26 

2.056 

0.079 

Lack  of  fit 

15 

1.578 

0.105  2.42 

0.072 

Pure  error 

11 

0.478 

0.043 

Total 

28 

40.586 

13  rows  with  no  replicates 

Source  DF  Seq  SS 

Log(M)  1  37.057 

Log(T)  1  1.473 

No  evidence  of  lack  of  fit  (P  >  0.1) 

Establish  the  weight  for  the  second  regression  run. 
Let  W2  =  2/(1  +  (TRES1)**2) 


Regression  analysis:  log(LD50)  versus  log(M), 
log(T) 

Weighted  regression  analysis  using  weights  in 
W2: 

The  regression  equation  is: 

Log(LDSo)  =  0.548  +  0.873  log(M)  +  0.606  log(T) 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

0.5479 

0.1505 

3.64 

0.001 

Log(M) 

0.87308 

0.03153 

27.69 

0.000 

1.1 

Log(T) 

0.6060 

0.1010 

6.00 

0.000 

1.1 

S  =  0.2327 
R-Sq  =  97.4% 
R-Sq(adj)  =  97.1% 
PRESS  =  1.70162 
R-Sq(pred)  =  96.80% 


Analysis  of  variance 


Source 

DF 

SS 

MS 

F 

P 

Regression 

2 

51.785 

25.893 

478.14 

0.000 

Residual  error 

26 

1.408 

0.054 

Lack  of  fit 

15 

0.930 

0.062 

1.43 

0.279 

Pure  error 

11 

0.478 

0.043 

Total 

28 

53.193 

1 3  rows  with  no  replicates 

Source  DF  Seq  SS 

Log(M)  1  49.837 

Log(T)  1  1.948 

Durbin-Watson  statistic  =  1.80 
No  evidence  of  lack  of  fit  (P  >  0.1) 

Establish  the  weight  for  the  third  regression  run. 
Let  W3  -  Pre  =  2/(1  +  (TRES3)**2) 

Let  W3  =  (W3  -  Pre  +  W2)/2 
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Regression  analysis:  log(LD50)  versus  log(M), 
log(T) 

Weighted  regression  analysis  using  weights  in 
W3: 

The  regression  equation  is: 


Log(LD50)  = 

0.547  +  0.872  log(M)  + 

0.608  log(T) 

Predictor 

Coef 

SE  Coef  T 

P 

VIF 

Constant 

0.5472 

0.1551  3.53 

0.002 

Log(M) 

0.87183 

0.03344  26.07 

0.000 

1.1 

Log(T) 

0.6077 

0.1039  5.85 

0.000 

1.1 

S  =  0.2399 

R-Sq  =  97.1% 

R-Sq(adj)  =  96.8% 

PRESS  =  1.83254 

R-Sq(pred)  = 

96.40% 

Analysis  of  variance 

Source 

DF 

SS  MS 

F 

P 

Regression 

2  ■ 

49.448  24.724 

429.52 

0.000 

Residual  error 

26 

1.497  0.058 

Lack  of  fit 

15 

1.035  0.069 

1.65 

0.205 

Pure  error 

11 

0.461  0.042 

Total 

28  : 

50.945 

13  rows  with  no  replicates 

Source  DF  Seq  SS 

Log(M)  1  47.478 

Log(T)  1  1.971 

Durbin- Watson  statistic  =  1.81 
No  evidence  of  lack  of  fit  (P  >  0.1) 

Calculate  the  sum  of  squared  differences  of  the 
last  two  weights,  and  compare  to  tolerance  level. 

Let  DIFF  (W3  -  W2)**2  =  (W3  -  W2)**2 

Sum  of  DIFF  (W3  -  W2)**2  =  0.48642 

The  first  difference  of  successive  weights  occurred 
after  the  third  run  was  made.  The  difference  of  the 
second  weight  (used  for  the  2nd  regression  run  but 
produced  by  the  1st  regression  model)  and  the 
third  weight  (used  for  the  3rd  regression  run  but 
produced  by  the  2nd  regression  model)  was  squared 


for  each  observation.  The  sum  of  the  squared 
differences  (SSD)  was  found  to  equal  0.53764. 
Iterations  of  model  building  will  be  continued 
until  the  SSD  does  not  exceed  a  predetermined 
“tolerance.” 

As  a  result  of  the  7th  run,  the  SSD  attained  a  value 
of  0.000572.  Several  more  runs  were  made  to 
acquire  a  finer  level  of  accuracy.  The  final 
regression  run  is  provided  below. 

Regression  analysis:  log(LDso)  versus  log(M), 
log(T) 

Weighted  regression  analysis  using  weights  in  W 
30: 

The  regression  equation  is: 

Log(LDSo)  =  0.515  +  0.869  log(M)  +  0.630  log(T) 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

0.5147 

0.1526 

3.37 

0.002 

Log(M) 

0.86908 

0.03347 

25.97 

0.000 

1.1 

Log(T) 

0.6296 

0.1015 

6.20 

0.000 

1.1 

S  =  0.2373 

R-Sq  =  97.1% 

R-Sq(adj)  = 

96.9% 

PRESS  =  1.79176 

R-Sq(pred) 

=  96.43% 

Analysis  of  variance 

Source 

DF 

SS 

MS 

F 

P 

Regression 

2 

48.747  24.373 

432.79 

0.000 

Residual  error  26 

1.4642 

0.056 

Lack  of  fit  1 5 

1.012 

0.067 

1.64 

0.206 

Pure  error  1 1 

0.453  0.041 

Total  28 

50.211 

1 3  rows  with  no  replicates 

Source  DF 

Sea  SS 

Log(M)  1 

46.581 

Log(T)  1 

2.165 

Durbin-Watson  statistic  =1.81 
No  evidence  of  lack  of  fit  (P  >  0.1) 
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Predicted  values  for  new 

New 

Fit 

SE  Fit 

Obs 

1 

2.3078 

0.1555 

2 

2.7479 

0.1016 

3 

3.0483 

0.0815 

4 

3.2378 

0.0821 

5 

3.4273 

0.0933 

6 

3.6168 

0.1120 

7 

3.7277 

0.1251 

8 

3.8064 

0.1350 

9 

3.9959 

0.1606 

observations: 


95.0%  Cl 

95.0% 

Pi 

(1.9881, 

(1.7246, 

2.6275) 

2.8910) 

(2.5390, 

(2.2172, 

2.9567) 

3.2785) 

(2.8807, 

(2.5325, 

3.2158) 

3.5640) 

(3.0691, 

(2.7216, 

3.4065) 

3.7539) 

(3.2356, 

(2.9032, 

3.6191) 

3.9514) 

(3.3867, 

(3.0775, 

3.8470) 

4.1562) 

(3.4706, 

(3.1763, 

3.9848) 

4.2791) 

(3.5288, 

(3.2451, 

4.0839) 

4.3676) 

(3.6657, 

(3.4069, 

4.3261) 

4.5849) 

Tables  F-l  through  F-3  summarize  the  data  in 
Appendix  F. 


Values  of  predictors 

for  new  observations 

New  Obs 

log(M) 

login 

1 

1.85 

0.30 

2 

1.85 

1.00 

3 

1.85 

1.48 

4 

1.85 

1.78 

5 

1.85 

2.08 

6 

1.85 

2.38 

7 

1.85 

2.56 

8 

1.85 

2.68 

9 

1.85 

2.98 

Mean  square  error  =  0.0563 163 


“X-prime  X”  inverse  matrix: 


0.413737 

0.029592 

-0.264750 

0.029592 

0.019892 

-0.016512 

-0.264750 

-0.016512 

0.183084 

Variance-covariance  matrix: 


0.023300 

0.001667 

-0.014910 

0.001667 

0.001120 

-0.000930 

-0.014910 

-0.000930 

0.010311 
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Table  F-1.  Raw  data,  input  quantities,  and  output  results 


MODEL  INPUT 

MODEL  OUTPUT 

Species 

Time  (min) 

Minute  Volume 
(m3/min) 

Observed 

LCTso 

Mass  (kg) 

Observed  LD50 

Observed  Log 

(LD50) 

Log(Time) 

Log(Mass) 

Last  Applied 

Weight 

Standardized 

Residuals 

Studentized 

Residuals 

Predicted  log 

(LD5q) 

Predicted  LD50 

Mouse 

10 

0.000031 

8752 

0.0207 

0.271 

-0.56653 

1.00000 

-1.68403 

0.94547 

-1.05376 

-1.05610 

-0.31923 

0.48 

Mouse 

10 

0.000031 

13150 

0.0207 

0.408 

-0.38971 

1.00000 

-1.68403 

1.71066 

-0.41800 

-0.41126 

-0.31923 

0.48 

Mouse 

10 

0.000031 

15200 

0.0207 

0.471 

-0.32679 

1.00000 

-1.68403 

1.99537 

-0.04911 

-0.04815 

-0.31923 

0.48 

Mouse 

10 

0.000031 

18199 

0.0207 

0.564 

-0.24859 

1.00000 

-1.68403 

1.70972 

0.41878 

0.41204 

-0.31923 

0.48 

Mouse 

10 

0.000031 

19430 

0.0207 

0.602 

-0.22017 

1.00000 

-1.68403 

1.54100 

0.55328 

0.54576 

-0.31923 

0.48 

Mouse 

10 

0.000031 

30620 

0.0207 

0.949 

-0.02263 

1.00000 

-1.68403 

0.82988 

1.17818 

1.18743 

-0.31923 

0.48 

Mouse 

11 

0.000031 

9245 

0.0207 

0.287 

-0.54273 

1.04139 

-1.68403 

0.94091 

-1.05839 

-1.06094 

-0.29317 

0.51 

Mouse 

30 

0.000031 

10682 

0.0207 

0.331 

-0.47999 

1.47712 

-1.68403 

0.58394 

-1.51625 

-1.55724 

-0.01883 

0.96 

Mouse 

30 

0.000031 

49300 

0.0207 

1.528 

0.18421 

1.47712 

-1.68403 

1.08060 

0.92506 

0.92240 

-0.01883 

0.96 

Mouse 

55 

0.000031 

27096 

0.0207 

0.840 

-0.07573 

1.74036 

-1.68403 

1.00725 

-0.99305 

-0.99278 

0.14690 

1.40 

Mouse 

60 

0.000031 

23822 

0.0207 

0.738 

-0.13166 

1.77815 

-1.68403 

0.81079 

-1.20036 

-1.21108 

0.17070 

1.48 

Rat 

5 

0.000203 

77850 

0.2610 

15.804 

1.19875 

0.69897 

-0.58336 

0.37091 

1.97110 

2.09574 

0.44781 

2.80 

Rat 

10 

0.000203 

61160 

0.2610 

12.415 

1.09396 

1.00000 

-0.58336 

0.59063 

1.50514 

1.54474 

0.63734 

4.34 

Rat 

30 

0.000203 

54810 

0.2610 

11.126 

1.04636 

1.47712 

-0.58336 

1.51078 

0.57659 

0.56905 

0.93774 

8.66 

Rat 

60 

0.000203 

50947 

0.2610 

10.342 

1.01461 

1.77815 

-0.58336 

1.47646 

-0.60301 

-0.59548 

1.12727 

13.41 

Rat 

60 

0.000203 

83820 

0.2610 

17.015 

1.23084 

1.77815 

-0.58336 

1.52633 

0.56461 

0.55707 

1.12727 

13.41 

G  Pig 

180 

0.000182 

290000 

0.3230 

52.780 

1.72247 

2.25527 

-0.49080 

1.00815 

0.99220 

0.99189 

1.50811 

32.22 

Rabbit 

30 

0.001453 

21580 

2.7700 

31.356 

1.49632 

1.47712 

0.44248 

0.76964 

-1.25005 

-1.26436 

1.82927 

67.49 

Rabbit 

30 

0.001453 

30479 

2.7700 

44.286 

1.64627 

1.47712 

0.44248 

1.16420 

-0.85193 

-0.84730 

1.82927 

67.49 

Rabbit 

190 

0.001453 

300000 

2.7700 

435.900 

2.63939 

2.27875 

0.44248 

0.79205 

1.22266 

1.23494 

2.33398 

215.76 

Cat 

235 

0.000845 

390000 

3.3600 

329.550 

2.51792 

2.37107 

0.52634 

1.77546 

0.36176 

0.35563 

2.46498 

291.73 

Dog 

10 

0.004230 

58340 

13.0000 

246.778 

2.39231 

1.00000 

1.11394 

0.85391 

1.15097 

1.15852 

2.11243 

129.55 

Dog 

30 

0.004230 

47010 

13.0000 

198.852 

2.29853 

1.47712 

1.11394 

1.46279 

-0.61352 

-0.60601 

2.41283 

258.72 

Dog 

30 

0.004230 

49380 

13.0000 

208.877 

2.31989 

1.47712 

1.11394 

1.58236 

-0.52118 

-0.51375 

2.41283 

258.72 

Dog 

30 

0.004230 

75000 

13.0000 

317.250 

2.50140 

1.47712 

1.11394 

1.60787 

0.50119 

0.49384 

2.41283 

258.72 

Dog 

30 

0.004230 

90000 

13.0000 

380.700 

2.58058 

1.47712 

1.11394 

1.21177 

0.81200 

0.80652 

2.41283 

258.72 

Goat 

8.5 

0.009600 

16500 

36.9000 

158.400 

2.19976 

0.92942 

1.56703 

0.87852 

-1.12388 

-1.12984 

2.46176 

289.57 

Goat 

18.5 

0.009600 

51500 

36.9000 

494.400 

2.69408 

1.26717 

1.56703 

1.96713 

0.13179 

0.12927 

2.67441 

472.51 

Sheep 

30 

0.013900 

24343 

52.6000 

338.368 

2.52939 

1.47712 

1.72099 

0.63676 

-1.43210 

-1.46319 

2.94040 

871.77 

F-4 


Table  F-2.  Human  dose  and  dosage  estimates  with  respect  to  exposure  time  from  allometric  fit 


TIME 

LOG(DOSE)  (LDso) 

DOSE  (LDso)  (mg) 

DOSAGE  (LCTso)  mg-min/m3 

(min) 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

2 

2.30779 

1.98810 

2.62748 

203 

97 

424 

13500 

6500 

28300 

10 

2.74786 

2.53901 

2.95671 

560 

346 

905 

37300 

23100 

60300 

30 

3.04826 

2.88071 

3.21581 

1118 

760 

1644 

74500 

50700 

110000 

60 

3.23779 

3.06905 

3.40652 

1729 

1172 

2550 

115000 

78200 

170000 

120 

3.42732 

3.23557 

3.61906 

2675 

1720 

4160 

178000 

115000 

277000 

240 

3.61685 

3.38673 

3.84696 

4139 

2436 

7030 

276000 

162000 

469000 

360 

3.72771 

3.47065 

3.98478 

5342 

2956 

9656 

356000 

197000 

644000 

480 

3.80638 

3.52882 

4.08393 

6403 

3379 

12132 

427000 

225000 

809000 

960 

3.99591 

3.66572 

4.32609 

9906 

4631 

21188 

660000 

309000 

1145000 

Table  F-3.  Human  concentration  estimates  with  respect  to 
exposure  time  from  allometric  fit 


TIME 

CONCENTRATION  (LCso) 
(mg/m3) 

CONCENTRATION  (LCso)  (ppm) 

(min) 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

2 

6770 

3240 

14100 

2340 

1120 

4880 

10 

3730 

2310 

6030 

1290 

800 

2080 

30 

2480 

1690 

3650 

860 

580 

1260 

60 

1920 

1300 

2830 

660 

450 

980 

120 

1490 

960 

2310 

510 

330 

800 

240 

1150 

680 

1950 

400 

230 

670 

360 

990 

550 

1790 

340 

190 

620 

480 

890 

470 

1690 

310 

160 

580 

960 

690 

320 

1190 

240 

110 

410 
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APPENDIX  G:  MINITAB®  SESSION  IRLS  ANALYSIS  FOR  LOG(LC50)  AS  A  FUNCTION  OF 
LOG(TIME),  LOG(MASS)  AND  LOG(MVRATIO) 


Regression  analysis:  log(LC5o)  versus  log(M), 
log(T) 

Unweighted  regression: 

The  regression  equation  is: 

log(LCso)  =  3.78  +  0.103  log(M)  -  0.393  log(T) 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

3.7846 

0.2024 

18.70 

0.000 

Log(M) 

0.10310 

0.04624 

2.23 

0.035 

1.0 

Log(T) 

-0.3932 

0.1360 

-2.89 

0.008 

1.0 

S  =  0.3037 
R-Sq  =  30.2% 
R-Sq(adj)  =  24.8% 
PRESS  =  3.08666 
R-Sq(pred)  =  10.15% 

Analysis  of  variance 


Source 

DF 

SS 

MS 

F 

P 

Regression 

2 

1.03660 

0.51830 

5.62 

0.009 

Residual  error 

26 

2.39859 

0.09225 

Lack  of  fit 

15 

1.92066 

0.12804 

2.95 

0.038 

Pure  error 

11 

0.47793 

0.04345 

Total 

28 

3.43518 

13  rows  with  no  replicates 

Source  DF  Seq  SS 

Log(M)  1  0.26498 

Log(T)  1  0.77161 

Durbin-Watson  statistic  =  1.69 

The  same  iterative  procedure  demonstrated  in 
Appendix  F,  and  described  in  Appendix  E,  was 
employed  here.  A  weight  was  calculated  for  each 
iteration,  and  the  process  was  curtailed  when  the 
individual  SSD  of  two  consecutive  sets  of  weights 
(from  two  consecutive  regression  runs)  became 
close  to  zero. 


Regression  analysis:  log(LCso)  versus  log(M), 
log(T) 

Weighted  regression  analysis  using  weights  in 
W30: 

The  regression  equation  is: 

log(LCso)  -  3.62  +  0.0976  log(M)  -  0.278  log(T) 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

3.6208 

0.1670 

21.68 

0.000 

Log(M)  0.09764 

0.03575 

2.73 

0.011 

1.1 

Log(T) 

■0.2780 

0.1125 

-2.47 

0.020 

1.1 

S  =  0.2554 

R-Sq  =  29.2% 
R-Sq(adj)  =  23.8% 
PRESS  =  2.08444 

R-Sq(pred)  = 

13.02% 

Analysis  of  variance 

Source 

DF 

SS 

MS 

F 

P 

Regression 

2 

0.69982 

0.34991 

5.36 

0.011 

Residual  error 

26 

1.69658 

0.06525 

Lack  of  fit 

15 

1.22493 

0.08166 

1.90 

0.142 

Pure  error 

11 

0.47165 

0.04288 

Total 

28 

2.39639 

13  rows  with  no  replicates 

Source  DF  Seq  SS 

Log(M)  1  0.30115 

Log(T)  1  0.39867 

Durbin-Watson  statistic  =  1.72 


The  final  run  is  depicted  below. 


G-1 


Predicted  values  for  new  observations: 


New 

Obs 

Fit 

SE  Fit 

95.0% 

ci 

95.0% 

PI 

1 

3.7172 

0.1685 

(3.3709, 

4.0636) 

(3.0882, 

4.3463) 

2 

3.5229 

0.1088 

(3.2994, 

3.7465) 

(2.9522, 

4.0936) 

3 

3.3903 

0.0873 

(3.2108, 

3.5697) 

(2.8353, 

3.9452) 

4 

3.3066 

0.0891 

(3.1235, 

3.4897) 

(2.7505, 

3.8626) 

5 

3.2229 

0.1027 

(3.0118, 

3.4339) 

(2.6570, 

3.7888) 

6 

3.1392 

0.1242 

(2.8838, 

3.3945) 

(2.5553, 

3.7231) 

7 

3.0902 

0.1392 

(2.8042, 

3.3762) 

(2.4923, 

3.6881) 

8 

3.0555 

0.1504 

(2.7463, 

3.3647) 

(2.4461, 

3.6648) 

9  2.9718  0.1792  (2.6034, 

3.3401) 

No  evidence  of  lack  of  fit  (P  >  0.1) 

(2.3304, 

3.6132) 

Values  of  predictors  for  new  observations: 

New  Obs  log(M)  log(T) 


1 

1.85 

0.30 

2 

1.85 

1.00 

3 

1.85 

1.48 

4 

1.85 

1.78 

5 

1.85 

2.08 

6 

1.85 

2.38 

7 

1.85 

2.56 

8 

1.85 

2.68 

9 

1.85 

2.98 

Mean  square  error  =  0.0652529 


“X-prime  X”  inverse  matrix: 


0.427454 

0.029429 

-0.277259 

0.029429 

0.019586 

-0.016489 

-0.277259 

-0.016489 

0.193902 

Variance-covariance  matrix: 


0.027893 

0.001920 

-0.018092 

0.001920 

0.001278 

-0.001076 

-0.018092 

-0.001076 

0.012653 

Regression  analysis:  log(LCso)  versus  log(M), 
log(MVratio),  log(T) 

Unweighted  regression: 


The  regression  equation  is: 
log(LC50)  =  4.24  -  0.0098  log(M) 
log(MVratio)  -  0.574  log(T) 

-2.85 

Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

4.2391 

0.2393 

17.72 

0.000 

Log(M) 

-0.00976 

0.05681 

-0.17 

0.865 

2.0 

Log(MVratio) 

-2.8534 

0.9962 

-2.86 

0.008 

2.5 

Log(T) 

-0.5742 

0.1359 

-4.23 

0.000 

1.3 

S  =  0.2688 
R-Sq  =  47.4% 
R-Sq(adj)  -41.1% 
PRESS  =  3.15099 
R-Sq(pred)  =  8.27% 

Analysis  of  variance 


Source 

DF 

SS 

MS 

f  E 

Regression 

3 

1.62924 

0.54308 

7.52  0.001 

Residual  error 

25 

1.80594 

0.07224 

Lack  of  fit 

14 

1.32802 

0.09486 

2.18  0.099 

Pure  error 

11 

0.47793 

0.04345 

Total 

28 

3.43518 

13  rows  with  no  replicates 


Source 

DF 

Sea  SS 

Log(M) 

1 

0.26498 

Log(MVratio) 

1 

0.07459 

Log(T) 

1 

1.28966 

The  inclusion  of  species  mass,  specifically  the 
variable  log(Mass),  does  not  contribute  to  the 
regression  equation.  Log(Mass)  has  attained  a  high 
p-value,  indicating  minimal  or  no  value  in 
predicting  log(LCso)  given  the  other  variables  in 
the  model.  The  presence  of  log(MVratio)  negates 
the  need  for  the  mass  variable  to  also  be  included. 
Additional  runs  are  made  without  log(Mass).  See 
Table  G-l  for  the  concentration,  dosage,  and  dose 
estimates  regression  with  log(LC5o)  versus  log(M) 
and  log(T)  data. 
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Table  G-1.  Concentration,  dosage,  and  dose  estimates  regression  with  log(LC50)  versus  log(M) 
and  log(T) 


TIME  (min) 

LOG  [CONCENTRATION  (LCso)] 

(mg/m3) 

CONCENTRATION  (LCso) 

(mg/m3) 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

2 

3.71724 

3.37091 

4.06358 

5210 

2350 

11580 

10 

3.52291 

3.29935 

3.74646 

3330 

1990 

5580 

30 

3.39025 

3.21078 

3.56972 

2460 

1620 

3710 

60 

3.30655 

3.12345 

3.48965 

2030 

1330 

3090 

120 

3.22286 

3.01184 

3.43388 

1670 

1030 

2720 

240 

3.13916 

2.88378 

3.39455 

1380 

770 

2480 

360 

3.09020 

2.80416 

3.37624 

1239 

640 

2380 

480 

3.05546 

2.74627 

3.36466 

1140 

560 

2320 

960 

2.97177 

2.60343 

3.34011 

940 

400 

2190 

TIME  /mini 

LOG  [CONCENTRATION  (LCso)] 

(ppm) 

DOSAGE  (LCTso) 

(mg-min/m 3) 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

2 

1800 

810 

3990 

10400 

4700 

23200 

10 

1150 

690 

1920 

33300 

19900 

55800 

30 

850 

560 

1280 

73700 

48700 

111000 

60 

700 

460 

1070 

122000 

79700 

185000 

120 

580 

350 

940 

200000 

123000 

326000 

240 

480 

260 

860 

331000 

184000 

595000 

360 

420 

220 

820 

443000 

229000 

856000 

480 

390 

190 

800 

545000 

268000 

1111000 

960 

320 

140 

760 

900000 

385000 

2101000 

Regression  analysis:  log(LC5o)  versus 
log(MVratio),  log(T) 

Unweighted  regression: 

log(LC50)  =  4.23  -  2.73  log(MVratio)  -  0.570 
log(T) _ 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

4.2260 

0.2225 

18.99 

0.000 

Log(MVratio) 

-2.7347 

0.7041 

-3.88 

0.001 

1.3 

Log(T) 

-0.5696 

0.1307 

-4.36 

0.000 

1.3 

S  =  0.2637 
R-Sq  =  47.4% 


R-Sq(adj)  =  43.3% 
PRESS  =  2.53042 
R-Sq(pred)  =  26.34% 


Analysis  of  variance 


Source 

DF 

ss 

MS 

F 

P 

Regression 

2 

1.62711 

0.81355 

11.70 

0.000 

Residual  error 

26 

1.80808 

0.06954 

Lack  of  fit 

15 

1.33015 

0.08868 

2.04 

0.118 

Pure  error 

11 

0.47793 

0.04345 

Total 

28 

3.43518 

13  rows  with  no  replicates 
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Source 

DF 

Sea  SS 

Source 

DF 

Sea  SS 

Log(MVratio) 

1 

0.30663 

Log(MVratio) 

1 

0.40779 

Log(T) 

1 

1.32047 

Log(T) 

1 

1.46509 

Durbin-Watson  statistic  =  2.05 


Durbin-Watson  statistic  =  2.04 


No  evidence  of  lack  of  fit  (P  >  0.1) 


No  evidence  of  lack  of  fit  (P  >  0.1) 


The  same  iterative  procedure  demonstrated  in 
Appendix  F,  and  described  in  Appendix  E,  was 
employed  here.  A  weight  was  calculated  for  each 
iteration,  and  the  process  was  curtailed  when  the 
individual  squared  differences  (SSD)  of  two 
consecutive  sets  of  weights  (from  two  consecutive 
regression  runs)  became  close  to  zero. 

The  final  run  is  depicted  below. 

Regression  analysis:  log(LCso)  versus, 
log(MVratio),  log(T) 

Weighted  regression  analysis  using  weights  in 
W30 

The  regression  equation  is: 

log(LC50)  =  4.31  -  3.03  log(MVratio)  -  0.627 

log(T) _ 


Predictor 

Coef 

SE  Coef 

1 

P 

VIF 

Constant 

4.3117 

0.1913 

22.54 

0.000 

Log(MVratio) 

-3.0292 

0.5721 

-5.29 

0.000 

1.3 

Log(T) 

-0.6272 

0.1131 

-5.55 

0.000 

1.3 

S  =  0.2182 
R-Sq  =  60.2% 


Predicted  values  for  new  observations: 


New 

Obs 

1 

2 

3 

4 

5 

6 
7 


9 


Fit 

SE  Fit 

95.0% 

ci 

95.0% 

PI 

4.1229 

0.1591 

(3.7959, 

4.4499) 

(3.5677, 

4.6780) 

3.6845 

0.0898 

(3.4999, 

3.8690) 

(3.1994, 

4.1696) 

3.3852 

0.0589 

(3.2642, 

3.5062) 

(2.9206, 

3.8498) 

3.1964 

0.0597 

(3.0737, 

3.3191) 

(2.7313, 

3.6615) 

3.0076 

0.0773 

(2.8487, 

3.1665) 

(2.5316, 

3.4835) 

2.8187 

0.1035 

(2.6060, 

3.0315) 

(2.3223, 

3.3152) 

2.7083 

0.1206 

(2.4603, 

2.9563) 

(2.1957, 

3.2209) 

2.6299 

0.1333 

(2.3560, 

2.9039) 

(2.1043, 

3.1556) 

2.4411 

0.1647 

(2.1026, 

2.7797) 

(1.8791, 

3.0031) 

Values  of  predictors  for  new  observations: 

New  Obs  log(MVratio)  log(T) 


R-Sq(adj)  =  57.1% 
PRESS  =  1.69334 
R-Sq(pred)  =  45.57% 

Analysis  of  variance 


Source 

DF 

SS 

MS 

F 

P 

Regression 

2 

1.87288 

0.93644 

19.66 

0.000 

Residual 

26 

1.23841 

0.04763 

error 

Lack  of  fit 

15 

0.80505 

0.05367 

1.36 

0.306 

Pure  error 

11 

0.43336 

0.03940 

Total 

28 

3.11129 

1 

0.000000 

0.30 

2 

0.000000 

1.00 

3 

0.000000 

1.48 

4 

0.000000 

1.78 

5 

0.000000 

2.08 

6 

0.000000 

2.38 

7 

0.000000 

2.56 

8 

0.000000 

2.68 

9 

0.000000 

2.98 

Mean  square  error  =  0.04763 12 

“X-prime 

X”  inverse  matrix: 

0.76816 

-1.53853 

-0.43372 

-1.53853 

6.87194 

0.67487 

-0.43372 

0.67487 

0.26853 

13  rows  with  no  replicates 
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Variance-covariance  matrix: 


0.036588 

-0.073282 

-0.020659 

-0.073282 

0.327319 

0.032145 

-0.020659 

0.032145 

0.012790 

See  Table  G-2  for  the  concentration,  dosage,  and 
dose  estimates  regression  with  log(LC50)  versus 
log(MVratio)  and  log(T)  data. 


Table  G-2.  Concentration,  dosage,  and  dose  estimates  regression  with  log(LC50)  versus 
log(MVratio)  and  log(T) 


TIME  (mini 

LOG  [CONCENTRATION  (LCso)] 

(mg/m3) 

CONCENTRATION  (LCso) 

(mg/m3) 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

2 

4.12288 

3.79587 

4.44990 

13300 

6250 

28200 

10 

3.68446 

3.49990 

3.86902 

4840 

3160 

7400 

30 

3.38520 

3.26419 

3.50620 

2430 

1840 

3210 

60 

3.19638 

3.07371 

3.31905 

1570 

1180 

2080 

120 

3.00756 

2.84866 

3.16647 

1020 

710 

1470 

240 

2.81875 

2.60604 

3.03146 

660 

400 

1080 

360 

2.70830 

2.46034 

2.95625 

510 

290 

900 

480 

2.62993 

2.35601 

2.90386 

430 

230 

800 

960 

2.44112 

2.10256 

2.77967 

280 

130 

600 

TIME  (mini 

CONCENTRATION  (LCso)] 

(ppm) 

DOSAGE  (LCTso) 

(mg-min/m 3) 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

POINT 

ESTIMATE 

LOWER 

BOUND 

UPPER 

BOUND 

2 

4580 

2160 

9720 

26500 

12500 

56400 

10 

1670 

1090 

2550 

48400 

31600 

74000 

30 

840 

630 

1110 

72800 

55100 

96200 

60 

540 

410 

720 

94300 

71100 

125000 

120 

350 

240 

510 

122000 

84700 

176000 

240 

230 

140 

370 

158000 

96900 

258000 

360 

180 

100 

310 

184000 

104000 

326000 

480 

150 

80 

280 

205000 

109000 

385000 

960 

100 

40 

210 

265000 

122000 

578000 
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APPENDIX  H:  EXTRAPOLATION  OF  HUMAN  CHLORINE  LETHALITY  ESTIMATE  FROM  A 
MILITARY  SUBPOPULATION  TO  GENERAL  POPULATION  BASIS  USING  THE  METHOD 
OF  CROSIER  (2007) — John  J.  Bray  and  Douglas  R.  Sommerville 


H1.0  Introduction 

It  is  widely  believed  that  the  general  population  is 
more  susceptible  to  toxicants  than  the  military 
population  is  and  that  the  general  population  has 
more  variability  in  susceptibility  to  toxicants  than 
the  military  subpopulation  does.1'9  In  the  absence 
of  data  relevant  to  the  soldier-to-civilian 
adjustment,  a  subjective  estimate  must  be  used.  In 
a  series  of  three  technical  reports,1'3  ECBC  has 
developed  a  mathematical  method  for  the 
conversion  from  one  population  basis  to  another, 
with  Crosier3  presenting  the  final  version  of  the 
method.  The  premise  of  the  approach  is  that 
mathematically  the  distribution  of  log(doses)  for  a 
healthy  subpopulation  is  located  completely  within 
the  distribution  formed  by  the  general  population 
(i.e.,  the  smaller  bell  curve  is  inside  the  larger  bell 
curve).  Previously,  Sommerville  et  al 9  formulated 
toxicity  estimates  for  chemical  warfare  agents 
(CWAs)  using  an  earlier  version2  of  the 
extrapolation  model.  The  reader  is  referred  to 
Crosier3  for  a  more  detailed  description  of  the 
method. 

H2.0  Model  Parameter  Inputs 

The  Crosier  method3  requires  the  following 
parameter  inputs  are  required  when  extrapolating 
toxicity  estimates  from  a  subpopulation  basis  to 
that  of  a  general  population: 

(1)  Size  of  the  subpopulation  (as  a  percent  of  the 
whole  population) 

(2)  Median  effective  dose/dosage 

(3)  Probit  slope  (base  10),  annotated  mSUb 

It  is  assumed  that  the  military  subpopulation 
comprises  30%  of  the  total  population.  The 
military  subpopulation  is  defined  as  anyone  in  the 
general  population  who  meets  the  age,  health,  and 
physical  fitness  requirements  for  military  service. 
The  30%  figure  is  subject  to  debate,  and  Crosier3 
does  provide  solutions  for  percent  sizes  from  5%  to 
75%.  If  a  smaller  size  is  chosen,  this  will  increase 
the  difference  in  median  effective  dosages  between 
subpopulation  and  whole  population,  while  the 
reverse  is  true  for  a  larger  size. 

From  Section  7.4. 1.4,  the  military  subpopulation 
probit  slope  for  chlorine  inhalation  lethality  was 


estimated  to  equal  8.0,  and  the  LCT50  was 
estimated  to  equal  13,500  mg-min/m3. 

H3.0  Model  Assumptions 

Crosier3  modeled  the  subpopulation  as  either  being 
based  upon  a  double  truncation  or  single  truncation 
of  the  whole  population.  The  basis  for  use  of 
double  truncation  is  that  age  is  the  covariate,  and 
young  adults  correspond  to  some  age  range,  say 
18  yr  to  35  yr.  As  pointed  out  by  Crosier,3  using 
double  truncation  will  result  in  children  being 
more  resistant  to  toxicants  than  young  adults  are, 
which  is  not  acceptable  for  risk  assessment.  For 
single  truncation,  the  basis  is  health,  and  it  is 
assumed  that  the  military  will  not  reject  someone 
for  being  too  healthy.  Single  truncation  is  a  more 
logical  assumption  for  the  present  application. 

There  is  a  region  of  feasible  values  for  a 
subpopulation’s  mean  log(dose)  and  standard 
deviation  relative  to  the  whole  population  where 
the  distribution  of  the  subpopulation’s  log(doses) 
can  still  be  adequately  represented  by  a  normal 
distribution.  This  is  demonstrated  in  Crosier’s3 
Figure  5.  For  risk  assessment  purposes,  the 
maximum  difference  in  median  effective  dosages 
is  desired,  and  for  the  single  truncation 
subpopulation,  this  point  is  represented  by  a  circle 
in  his  Figure  5. 

H4.0  Model  Outputs 

From  Crosier,3  the  following  parameter  values 
were  obtained  for  a  single  truncation  subpopulation 
with  a  size  of  30%  of  the  general  population: 

(1)  mean  8t  equals  0.900 

(2)  sT  (standard  deviation  factor)  equals  0.744 
These  parameters  are  then  used  in  the  following 
equations  to  arrive  at  parameter  estimates  for  the 
general  population  for  a  2-min  inhalation  exposure 
to  chlorine  vapor. 

"Uo/e  =  msUb  =  (0.744)(8.0)  =  5.952  (HI) 


Ratio  =  (-1)  — — — 
"W 


(  ^(0-900) 

^  ’  (5.952) 


(-0.151) 


(H2) 


H-1 


(LCT5o  )wfioJe  =  10Ra"°  (LCT50)suh 
=  10(-°151)  (13500) 

=  9500  mg-min/m3 


Yn  =  (-31.52)  +  (2.91)log(C2  75T ) 
(H3)  or 

L(C2  75T)so  =  6.79x1010 


A  toxic  load  exponent  value  of  2.75  was  then  used  ,  275  >, 

to  extrapolate  to  other  LCT50  values  from  the  two  YfJ  ~  (_22-698)  +  (2.18)log(C  7) 

minute  value.  For  other  percent  effect  levels,  a  or 

round  probit  slope  value  of  6.0  is  recommended.  l(C2-75t)5o  =  2.58 xIO10 

See  Table  H-l  for  a  listing  of  subpopulation 

toxicity  values  for  several  selected  exposure 

durations.  The  toxic  load  equations  are  shown 

below  for  both  military  (eq  F14)  and  general 

(eq  H5)  populations. 


(H4) 


(H5) 


Table  H-1.  Chlorine  inhalation  median  lethal  dosages  and  probit  slope  for  the  general 
population 


Agent! 

Chlorine 

SINGLE  TRUNCATION 

CALCULATIONS 

Nomenclature: 

Cl2 

Standard  Deviation  =0.744 

(for  single  truncation) 

ITl  (slope)  whole  population  =5.952 

(subpopulation  slope  x  standard  deviation) 

Z  (for  mean  single  truncation)  =  0.900 

Ratio  (-z/m  whole )  =-0.151 

Conversion  Factor  .  -,n„ 

=  0.706 

[antilog  (ratio)] 

Type  of  Exposure: 

Inhalation/Ocular 

Form: 

Vapor 

Level: 

Lethal 

Toxic  Load 

Exponent: 

2.75 

Temperature: 

None  Specified 

MILITARY 

SUBPOPULATION 

WHOLE  POPULATION 

SINGLE  TRUNCATION 

EXPOSURE 
DURATION  (MIN) 

PROBIT  SLOP 

E  =  8.0 

PROBIT  SLOPE  =  6.0 

LCTso 

(mg-min/m3) 

LCso 

(mg/m3) 

LCTso 

(mg-min/m3) 

LCso 

(mg/m3) 

2 

13500 

6750 

9500 

4750 

10 

37600 

3760 

26400 

2640 

30 

75700 

2520 

53200 

1770 

60 

118000 

1970 

82700 

1380 

120 

183000 

1530 

129000 

1080 

240 

284000 

1180 

200000 

830 

360 

368000 

1020 

259000 

720 

480 

442000 

920 

311000 

650 

960 

687000 

720 

483000 

500 
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APPENDIX  I:  CHLORINE  HAZARD  PREDICTION— Raymond  E.  Jablonski  and  Douglas  R. 
Sommerville 

11.0  Introduction 


For  this  report,  a  series  of  computer  simulations 
were  conducted  of  a  large  chlorine  spill  release  for 
eventual  comparison  to  the  toxicity  methodologies 
discussed  in  this  report.  Initially,  two  atmospheric 
transport  and  dispersion  (ATD)  models  were  used: 
the  EPA/NOAA  Areal  Locations  of  Hazardous 
Atmospheres  (ALOHA)1  and  Defense  Threat 
Reduction  Agency’s  (DTRA’s)  Hazard  Prediction 
Assessment  Capability  (HP AC)  models.2  Both  of 
these  models  were  part  of  a  recent  evaluation  of 
dense  gas  dispersion  models  (along  with  four  other 
models)  involving  the  simulation  of  three  railroad 
accidents  involving  the  bulk  release  of  chlorine 
vapor.3  The  predictions  from  both  ALOHA  and 
HPAC  were  found  to  be  in  “good  agreement”  with 
the  predictions  of  the  other  four  ATD  models. 
Only  the  HPAC  model  provided  numeric  output  of 
concentration  time  history  needed  for  the  toxic 
load  analysis  discussed  in  Section  7.5  in  the  main 
body  of  the  report.  A  summary  of  the  simulations 
and  results  produced  by  both  models  are  discussed 
in  this  appendix. 

12.0  Model  Description 

The  following  model  descriptions  are  largely 
extracted  from  their  respective  user’s  manuals.1’2 

12.1  HPAC 

The  Hazard  Prediction  and  Assessment  Capability 
(HPAC)2  automated  software  system  provides  the 
means  to  accurately  predict  the  effects  of 
hazardous  material  releases  into  the  atmosphere 
and  its  impact  on  civilian  and  military  populations. 
The  system  uses  integrated  source  terms,  high- 
resolution  weather  forecasts  and  particulate 
transport  analyses  to  model  hazard  areas  produced 
by  military  or  terrorist  incidents  and  industrial 
accidents.  HPAC  and  its  companion  tools  [e.g., 
Consequence  Assessment  Toolset  (CATS)] 
provide  actionable  information  to  support 
consequence  assessment  and  management  of 
CBRNE  hazards,  the  impact  on  personnel  and 
operations,  and  resource  requirements  for  response 
and  recovery. 

The  transport  and  diffusion  sub-model  embedded 
within  HPAC  is  the  Second-order  Closure 
Integrated  Puff  (SCIPUFF)  model.  SCIPUFF  is  an 
atmospheric  dispersion  model  with  a  wide  range  of 


application.  The  turbulent  diffusion  parameter¬ 
ization  is  based  on  second-order  turbulence  closure 
theory,  which  relates  the  dispersion  rate  to  velocity 
fluctuation  statistics.  In  addition  to  the  average 
concentration  value,  the  closure  model  provides  a 
prediction  of  the  statistical  variance  in  the 
concentration  field  resulting  from  the  random 
fluctuations  in  the  wind  field.  The  variance  is  used 
to  estimate  a  probability  distribution  for  the 
predicted  value. 

SCIPUFF  uses  a  collection  of  Gaussian  puffs  to 
represent  an  arbitrary  three-dimensional,  time- 
dependent  concentration  field,  and  incorporates  an 
efficient  scheme  for  splitting  and  merging  puffs. 
Wind  shear  effects  are  accurately  modeled,  and 
puffs  are  split  when  they  grow  too  large  for  single¬ 
point  meteorology  to  be  representative.  These 
techniques  allow  the  puff  model  to  describe 
complex  flow  effects  on  dispersion,  such  as 
terrain-driven  circulations. 

SCIPUFF  has  been  developed  with  a  flexible 
interface,  to  describe  many  types  of  source 
geometries  and  material  properties.  Solid  particles, 
liquid  droplets,  and  gaseous  materials  are 
represented,  with  both  primary  and  secondary 
evaporation  mechanisms  that  produce  vapor  puffs 
as  the  droplets  evaporate  in  the  air  or  after 
deposition  on  the  ground.  Precipitation  washout 
effects  are  also  included  for  particles  and  droplets. 
SCIPUFF  describes  dynamic  effects  of  buoyant 
rise  due  to  thermal  release  or  lighter- than- air 
materials  and  also  the  effects  of  a  dense  cloud  near 
the  ground  surface.  The  model  also  uses  several 
types  of  meteorological  input,  including  surface 
and  upper  air  observations  or  three-dimensional 
grid  data.  Planetary  boundary  layer  turbulence  is 
represented  explicitly  in  terms  of  surface  heat  flux 
and  shear  stress  using  parameterized  profile 
shapes.  The  HPAC  system  also  has  embedded 
climatology  or  historical  weather  data  for  use  when 
planning  incidents  beyond  the  normal  time 
associated  with  credible  weather  forecast  data. 
One-kilometer  terrain  data  and  supporting  wind- 
flow  models  calculate  the  local  winds  field  in  the 
area  of  concern.  Other  weather  sources  are  also 
available  upon  request  to  DTRA.  The  HPAC 
system  can  also  help  answer  the  question,  “How 
good  is  the  prediction?”  by  providing  probabilistic 
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calculations.  The  hazard  area  feature  estimates  the 
weather  uncertainty  and  turbulence  effects  on 
possible  plume  trajectories  and  calculates  the  areas 
of  hazard  impact  and  the  degree  of  confidence  of 
the  prediction. 

12.2  ALOHA 

ALOHA1  is  a  computer  program  designed 
especially  for  use  by  people  responding  to 
chemical  releases,  as  well  as  for  emergency 
planning  and  training.  ALOHA  models  key 
hazards — toxicity,  flammability,  thermal  radiation 
(heat),  and  overpressure  (explosion  blast  force) — 
related  to  chemical  releases  that  result  in  toxic  gas 
dispersions  (i.e.,  how  a  toxic  gas  cloud  might 
disperse  in  the  atmosphere  after  an  accidental 
chemical  release),  fires,  and/or  explosions. 

ALOHA  runs  quickly  on  small  computers/laptops 
that  are  easily  transportable.  It  is  designed  to  be 
easy  to  use  so  that  a  user  can  operate  it 
successfully  during  high-pressure  situations.  Its 
chemical  library  contains  information  about  the 
physical  properties  of  approximately  1000 
common  hazardous  chemicals.  Its  computations 
represent  a  compromise  between  accuracy  and 
speed;  ALOHA  has  been  designed  to  produce  good 
results  quickly  enough  to  be  of  use  to  responders. 
ALOHA  is  designed  to  minimize  operator  error.  It 
checks  information  that  the  user  enters  and  warns 
the  user  when  he/she  makes  a  mistake.  ALOHA’s 
on-screen  help  offers  you  quick  access  to 
explanations  of  ALOHA’s  features  and  computa¬ 
tions,  as  well  as  background  information  to  help 
you  interpret  its  output. 

ALOHA  models  three  hazard  categories:  toxic  gas 
dispersion,  fires,  and  explosions.  ALOHA  employs 
several  different  models,  including  an  air  disper¬ 
sion  model  that  it  uses  to  estimate  the  movement 
and  dispersion  of  chemical  gas  clouds.  From  this 
model,  ALOHA  is  able  to  estimate  the  toxic  gas 
dispersion,  the  overpressure  values  from  a  vapor 
cloud  explosion,  or  the  flammable  areas  of  a  vapor 
cloud. 

12.3  Chlorine  Scenario  Modeled 

The  scenario  modeled  was  for  a  catastrophic 
release  of  either  90  tons  (-81,000  kg)  or  50  tons 
(-45,500  kg)  of  chlorine  liquid.  Ninety  tons  is  the 
typical  amount  for  a  standard  rail  tank  car. 
However,  a  release  of  50  tons  was  used  as  input 


into  HPAC,  since  this  is  the  largest  release 
reported  by  Mannan4  in  their  review  of  chlorine 
industrial/transportation  accidents.  Literature  com¬ 
mentary  on  the  downwind  location  of  fatalities  (see 
Section  4)  uses  this  accident  database  as  its 
reference.  Thus,  modeling  results  for  the  50-ton 
release  were  used  for  comparison  to  the  different 
toxicity  methodologies. 

Model  inputs  were  kept  fairly  simple  for  these 
simulations.  Flat  terrain  was  assumed.  Meteoro¬ 
logical  conditions  were  kept  at  a  steady-state,  fixed 
wind  condition  (same  wind  direction  and  wind 
speed  assumed  through  the  simulation).  The 
chlorine  release  was  assumed  to  be  a  catastrophic, 
instantaneous  release  of  the  liquid.  Both  ALOHA 
and  HPAC  make  estimates  of  the  amount  of 
chlorine  that  initially  pools,  but  all  liquid 
evaporated  very  quickly.  Since  meteorological 
conditions  can  produce  significant  differences  in 
hazard  prediction  results,  three  different  weather 
conditions  were  modeled.  These  three  conditions 
were: 

•  Low  wind  (1  m/s),  clear  sky,  nighttime  (2  a.m.), 
“stable”  condition  (Pasquill5  Stability  Category 
F) 

•  Moderate  wind  (5  m/s),  cloudy  day,  “neutral” 
condition  (Pasquill  Stability  Category  D) 

•  Low  wind  (2  m/s),  clear  sky,  daytime  (2  p.m.), 
“unstable”  condition  (Pasquill  Stability  Category 
B) 

12.4  Critical  Importance  of  Proper  Toxicity 
Effects  Definition 

One  of  the  major  concerns  within  the  chemicaP 
biological  defense  community  is  whether  current 
hazard  prediction  models  are  producing  reliable 
and  “accurate”  results.  A  number  of  current  models 
have  been  reviewed,  and  there  is  agreement  that 
the  methodology  used  within  these  models  is  “state 
of  the  art.”  Six  dense  gas  models  were  recently 
compared  to  three  actual  chlorine  railcar  accidents 
and  these  models  compared  well  with  one  another.3 
One  of  the  major  conclusions  of  the  study  was  that 
estimation  of  the  source  term  is  important  for  good 
results.  The  same  can  be  said  for  determining 
casualty  estimates.  Although  a  number  of  toxicity 
definitions  exist  (AEGLs,  ERPGs,  TEELs,  ICT50, 

LCT5o,  etc.),i  these  definitions  and  the  data  points 
they  represent  for  a  specific  chemical  must  be  in 
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sync  with  current  hazard  prediction  model 
calculations. 

An  important  problem  with  many  of  the  current 
toxicity  estimates  is  that  they  are  currently  used  in 
the  ATD  models  with  a  Haber’s  Rule  mindset — 
that  an  effective  dosage  (concentration-time  or  CT) 
is  constant  with  respect  to  time.  This  is  ironic, 
since  many  of  these  toxic  estimates  (including 
chlorine)  were  developed  with  the  acknowledge¬ 
ment  that  Haber’s  Rule  is  often  inadequate.  The 
toxic  load  model  is  a  possible  alternative  but  only 
if  the  result  is  used  with  a  toxic  load  mindset.  This 
issue  is  explored  further  in  Section  13.1. 

Based  on  results  from  this  brief  analysis,  there  may 
be  some  question  whether  the  necessary  synchro¬ 
nization  currently  exists.  The  following  sections 
provide  several  examples  and  identify  some 
questions  to  consider  for  further  investigation. 

13.0  Results 

13.1  HP  AC  Model  Runs 

13.1.1  Initial  Plots 

The  differences  in  plume  shape  and  downwind 
hazard  extent  are  depicted  in  the  HP  AC  contour 
plots  [Figure  1-1  through  Figure  1-3  (high 
resolution)  and  Figure  1-4  through  Figure  1-6  (low 
resolution)]  for  the  three  different  meteorological 
conditions.  Note  that  the  nighttime  release  will 
produce  the  largest  hazard  area,  while  the  daytime 
release  will  produce  the  smallest. 

Figure  1-1  through  Figure  1-3  (pages  1-4  and 
1-5)  display  dosage  contour  plots  using  LCT50 
values  from  eq  25  of  the  main  body  of  the 
report  (corresponding  to  exposure  durations 
from  2  min  to  60  min).  Figure  1-4  through 
Figure  1-6  (pages  1-5  and  1-6)  display  dosage 
contour  plots  across  a  range  of  toxicity  effects 
definitions.  These  include  the  old  military  LCT50 
and  ICT50  values6  (19,000  mg-min/m3  and  1800 
mg-min/m3,  respec-tively — which  are  independent 
of  exposure  duration)  and  a  series  of  chlorine 
AEGL  effects  levels7  for  different  exposure  times 
(10  min  to  60  min).  It  should  be  noted  that 
similar  AEGL 


definitions  (i.e.,  AEGL1,  AEGL2,  AEGL3)  at 
different  exposure  times  do  produce  significantly 
different  contour  areas.  These  plots  show  the 
criticality  of  proper  model  input  for  source  term, 
meteorological  data  and  toxicity  estimate 
definition. 

Another  important  observation  from  Figure  1-1 
through  Figure  1-6  is  that  the  plume  contours 
were  generated  by  HPAC  using  a  Haber’s  Rule 
mindset.  This  is  demonstrated  by  the  paradox  of 
having  the  plume  size  decrease  as  a  function  of  an 
increase  in  the  duration  basis  for  the  plotted  dosage 
contour  [i.e.,  the  plume  size  for  LCT50  (2  min)  is 
larger  than  LCT50  (60  min)  in  Figure  1-1  through 
Figure  1-3 — the  same  is  true  with  the  AEGL 
values  as  wel]  The  reason  for  this  is  that  HPAC 
currently  plots  the  dosage  contour  without 
reference  to  the  predicted  duration  value  that  was 
used  to  produce  the  dosage  value  for  that  contour.j 
In  all  likelihood,  the  duration  associated  with  the 
LCT50  (2  min)  contour  is  on  the  order  of  10  min  to 
60  min,  in  which  case  this  LCT50  value  is 
meaningless.  It  has  been  demonstrated  in  the  main 
body  of  this  report  and  elsewhere  that  the  LCT50 
for  chlorine  increases  as  a  function  of  exposure 
duration.  In  the  future,  a  solution  for  this  problem 
would  be  to  modify  HPAC  to  permit  the  plotting  of 
toxic  load  contours  (see  Section  13.1.2) 

13.1.2  HPAC  Sampler  Output 

Because  of  the  problem  in  the  HPAC  plotting 
routine  (see  Section  13.1.1),  model  results  for  five 
“sampler”  locations  were  also  produced  so  that  the 
necessary  calculations  could  be  made  independent 
of  the  misleading  HPAC  plots.  These  “samplers” 
were  located  0.2  km,  0.5  km,  1.0  km,  1.5  km,  and 
2.0  km  downwind  of  the  source  release.  Sampler 
output  data  from  HPAC  included  time  history  (at 
l-min  intervals),  mean  concentration  (in  kg/m3), 
concentration  variance  and  correlation  time  scale 
(in  seconds).  The  primary  output  needed  for  the 
toxicity  methodology  comparison  was  the  time 
history  and  mean  concentration  at  the  five 
locations.  An  example  of  the  numeric  output  data 
is  provided  in  Figure  1-7  (page  1-7). 


aAcute  Exposure  Guideline  Levels  (AEGLs);  Emergency  Response 

Exposure  Limits  (TEELs);  incapacitating  dosage  for  50%  of  the  exposed  population  (ICT50);  and  lethal  dosage  (CT)  for  50%  of 
the  exposed  population  (LCT50) 

Phone  conversation,  Mr.  Sommerville  with  Dr.  R.  Ian  Sykes,  L-3  Titan  Corporation,  Princeton,  NJ,  October  2007.  Author’s 
Note:  HPAC  can  presently  perform  toxic  load  calculations  for  casualty  assessment;  it  just  cannot  graphically  present  the  results  of 
such  calculations  in  an  easily  understandable  manner. 
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Figure  1-1.  HPAC  plumes  (high  resolution)  for  low  wind  (1  m/s),  clear  sky,  nighttime  (2  a.m.), 
“stable”  condition  (Pasquill  Category  F) 


Figure  1-2.  HPAC  plumes  (high  resolution)  for  moderate  wind  (5  m/s),  cloudy  day,  “neutral” 
condition  (Pasquill  Category  D) 
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Figure  1-3.  HPAC  plumes  (high  resolution)  for  low  wind  (2  m/s),  clear  sky,  daytime  (2  p.m.), 
“unstable”  condition  (Pasquill  Category  B) 


j  ...a 


Figure  1-4.  HPAC  plumes  (low  resolution)  for  low  wind  (1  m/s),  clear  sky,  nighttime  (2  a.m.), 
“stable”  condition  (Pasquill  Category  F) 
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Figure  1-5.  HPAC  plumes  (low  resolution)  for  moderate  wind  (5  m/s),  cloudy  day,  “neutral” 
condition  (Pasquill  Category  D) 
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Figure  1-6.  HPAC  plumes  (low  resolution)  for  low  wind  (2  m/s),  clear  sky,  daytime  (2  p.m.), 
“unstable”  condition  (Pasquill  Category  B) 
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Figure  1-7.  Example  of  numeric  output  data  from  HPAC 


In  Figure  1-8  through  Figure  1-10,  the 
concentration-time  profiles  associated  with  the 
three  meteorological  conditions  are  shown  as  a 
function  of  downwind  distance  along  the  centerline 
of  the  plume.  From  these  profiles,  the  accumulated 
toxic  load  was  calculated  via  the  rectangle  method 
for  numerical  integration  of  the  concentration-time 
profiles  using  the  below  algorithm. 

TL  =  \ToC"dt  »  +  C?_,)  At  (M) 

where  TL  is  the  total  toxic  load,  C  is  the  predicted 
mean  vapor  concentration  from  HPAC  (converted 
from  the  original  units  of  kg/m3  to  mg/m3),  T  is  the 
event  time  (in  minutes),  n  is  the  toxic  load 
exponent  and  At  equals  1  min. 

The  toxic  load  values  (for  TL  =  ( C2'15T ))  as  a 
function  of  time  are  shown  in  Figure  1-1 1  through 
Figure  1-13  corresponding  to  the  concentration¬ 
time  profiles  in  Figure  1-8  through  Figure  1-10). 
A  toxic  load  exponent  ( n )  of  2.75  was  chosen  since 
this  value  is  common  to  several  toxicity  estimates 
(Rijnmond  Report8/Harris  and  Moses, 9'k  ten  Berge 
and  van  Heemst,10  and  this  report’s  eq  25).  See 
Table  1-1  for  a  listing  of  the  lethality  estimates 
investigated  in  this  appendix  (which  were  extracted 


report).  For  comparison,  the  5%  lethal  toxic  load 
(or  LTL05)  estimates  from  eq  25  (upper  dashed  red 
line)  and  the  Rijnmond  Report  (bottom  dashed  red 
line)  are  also  shown  in  Figure  1-11  through 
Figure  1-13.  A  plume  contour  of  a  LTLxx  is 
independent  of  exposure  duration,  and  it  is 
recommended  that  this  useful  capability  be 
introduced  into  a  future  version  of  HPAC. 

The  behavior  of  the  TL  versus  time  profiles  is 
interesting.  The  total  TL  reaches  a  plateau  value  at 
about  the  instance  when  the  initial  “pulse”  of  the 
chlorine  cloud  passes  by  a  downwind  location. 


Table  1-1.  Human  lethality  equations  from  main 
body  of  this  report 


REFERENCE 

CONVERTED  EQUATION** 

Withers,  Lees 

Yn  (Normit)  =  -1 1 .697  +  1 .675  log(C  2  “  7) 

ten  Berge,  van 
Heemst 

Yn  (Normit)  =  -11.503  +  1.151  \og(C^T) 

Rijnmond  Report 
(Harris,  Moses) 

Yn  (Normit)  =  -1 8.799  +  1 .888  log(C 275  7) 

Equation  25 

Yn  (Normit)  =  -22.698  +  2.182  log(C 275  7) 

**The  concentration " 

C"  is  measured  in  mg/m3 

eHarris  and  Moses  and  the  Rijnmond  Report  have  identical  human  estimates  and  are  treated  as  synonymous  by  Mannan.4 
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Time  (minutes) 


Figure  1-8.  Mean  vapor  concentration  (2-min  TWA)  versus  time  and  downwind  distance — HPAC 
predictions  for  Pasquill  Category  F  release  scenario 


Figure  1-9.  Mean  vapor  concentration  (2-min  TWA)  versus  time  and  downwind  distance — HPAC 
predictions  for  Pasquill  Category  D  release  scenario 
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Figure  1-10.  Mean  vapor  concentration  (2-min  TWA)  versus  time  and  downwind  distance — HPAC 
predictions  for  Pasquill  Category  B  release  scenario 


Figure  1-11.  Accumulated  toxic  load  versus  time  and  downwind  distance — HPAC  predictions  for 
Pasquill  Category  F  release  scenario 
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Figure  1-12.  Accumulated  toxic  load  versus  time  and  downwind  distance — HPAC  predictions  for 
Pasquill  Category  D  release  scenario 
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Figure  1-13.  Accumulated  toxic  load  versus  time  and  downwind  distance — HPAC  predictions  for 
Pasquill  Category  B  release  scenario 
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Thus,  for  chlorine  (for  n  =  2.75),  most  of  the  lethal 
toxic  load  has  already  been  received  within  10  min 
to  30  min  (depending  on  atmospheric  conditions 
and  whether  lethal  toxic  load  levels  have  been 
reached)  post-arrival  of  the  initial  pulse.  For  sub- 
lethal  effects  (which  are  not  addressed  in  this 
report),  the  critical  post-arrival  period  is  probably 
longer  than  30  min  [particularly  if  n  (nonlethal)  is 
less  than  n  (lethal)].  Thus,  effective  sheltering  in 
place  measures  during  that  critical  period  is 
essential  if  evacuation  prior  to  the  cloud’s  arrival  is 
not  possible. 

The  percent  probability  of  lethality  was  then 
estimated  from  the  toxic  load  estimates  using 
several  of  the  probit  relationships4'8'12  from 
Table  1-2,  as  well  as  using  Flaber’s  Rule  based 
upon  the  2-min  LCT50  value  from  eq  25  (n  =  1,  kc 
=  6  and  LCT50  =  9500  mg-min/m3).  The  percent 
probability  values  (and  their  associated  normit 
values)  are  listed  in  Table  1-2  through  Table  1-4 
as  a  function  of  atmospheric  conditions  and  down¬ 


wind  distance.  Some  of  these  results  are  also 
plotted  in  Figures  7-6  to  7-8  in  the  main  body  of 
this  report.  Of  the  lethality  probit  relationships 
studied,  the  probability  of  lethality  versus 
downwind  distance  profiles  from  eq  25  and  the 
Rijnmond  Report8,9  are  the  most  consistent  with 
what  has  been  observed  historically  in  chlorine 
accidents.4  See  Section  7.5  of  the  main  body  of  the 
report  for  a  more  detailed  discussion. 

A  sensitivity  analysis  was  also  performed  on  how 
changes  in  the  toxic  load  exponent  and  probit  slope 
(concentration)  affect  the  probability  of  lethality 
versus  downwind  distance  profile.  The  concen¬ 
tration-time  profile  from  Figure  1-8  (page  1-8)  was 
used.  The  toxic  load  equation  parameter  inputs  and 
the  final  results  of  the  analysis  are  discussed  in 
Section  7.5.2  in  the  main  body  of  the  report. 


Table  1-2.  Lethality  probabilities  as  a  function  of  lethality  expression  and  distance  from  release 
point — nighttime  conditions  (Pasquill  Category  D) 


PRESENT  STUDY 
(EQ  25) 

RIJNMOND 

TEN  BERGE  & 
VAN  HEEMST 

WITHERS  & 
LEES 

HABER'S  RULE 

Lethal  Toxic  Load  (LTLso) 

2.582E+10 

9.059E+09 

9.861E+09 

9.662E+06 

9.500E+03 

Toxic  Load  Exponent  (n) 

2.75 

2.75 

2.75 

2.00 

1.00 

Probit  Slope  with  respect  to 
Toxic  Load  PS  (TL) 

2.18 

1.89 

1.15 

1.68 

6.00 

Probit  Slope  with  respect  to 
Concentration  PS  (CT) 

6.00 

5.19 

3.17 

3.35 

6.00 

1  H 

Distance  (kilometers) 

Prob  1%) 

Prob  (%) 

Prob  (%) 

Prob  (%) 

Prob  (%) 

0.2 

98.6 

99.7 

94.9 

99.7 

100.0 

0.5 

13.0 

45.4 

45.5 

81.3 

100.0 

1.0 

0.00 

0.28 

4.2 

22.8 

73.9 

1.5 

0.00 

0.00 

0.61 

5.6 

18.4 

2.0 

0.00 

0.00 

0.13 

1.7 

2.9 

1  1 

Distance  (kilometers) 

Norm  its 

Normits 

Normits 

Normits 

Normits 

0.2 

2.1989 

2.7644 

1.6397 

2.7351 

6.7584 

0.5 

-1.1263 

-0.1160 

-0.1130 

0.8881 

3.4215 

1.0 

-4.1952 

-2.7744 

-1.7305 

-0.7468 

0.6397 

1.5 

-5.6619 

-4.0450 

-2.5036 

-1.5884 

-0.9001 

2.0 

-6.6061 

-4.8629 

-3.0012 

-2.1257 

-1.8925 
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Table  1-3.  Lethality  probabilities  as  a  function  of  lethality  expression  and  distance  from  release 
point — nighttime  conditions  (Pasquill  Category  F) 


PRESENT  STUDY 
(EQ  25) 

RIJNMOND 

TEN  BERGE& 
VAN  HEEMST 

WITHERS  & 
LEES 

HABER'S  RULE 

Lethal  Toxic  Load  (LTLso) 

2.582E+10 

9.059E+09 

9.861  E+09 

9.662E+06 

9.500E+03 

Toxic  Load  Exponent  (n) 

2.75 

2.75 

2.75 

2.00 

1.00 

Probit  Slope  with  respect  to 

Toxic  Load  PS  (TL) 

2.18 

1.89 

1.15 

1.68 

6.00 

Probit  Slope  with  respect  to 
Concentration  PS  (CT) 

6.00 

5.19 

3.17 

3.35 

6.00 

l 

Distance  (kilometers) 

Prob  (%) 

Prob  (%) 

Prob  (%! 

Prob  1%) 

Prob  (%! 

0.2 

99.7 

99.9 

97.2 

99.9 

100.0 

0.5 

10.6 

41.2 

43.0 

85.4 

100.0 

1.0 

0.00 

0.15 

3.2 

25.7 

96.5 

1.5 

0.00 

0.00 

0.30 

5.8 

57.6 

2.0 

0.00 

0.00 

0.04 

1.5 

18.1 

1  H 

Distance  (kilometers) 

Normits 

Normits 

Normits 

Normits 

Normits 

0.2 

2.7047 

3.2026 

1.9063 

3.2669 

>  7.9347 

0.5 

-1.2486 

-0.2220 

-0.1774 

1.0523 

4.7386 

1.0 

-4.4246 

-2.9732 

-1.8514 

-0.6516 

1.8131 

1.5 

-6.1237 

-4.4450 

-2.7470 

-1.5723 

0.1907 

2.0 

-7.2132 

-5.3888 

-3.3213 

-2.1727 

-0.9113 

1-12 


Table  1-4.  Lethality  probabilities  as  a  function  of  lethality  expression  and  distance  from  release 
point — nighttime  conditions  (Pasquill  Category  B)) 


PRESENT  STUDY 
(EQ  25) 

RIJNMOND 

TEN  BERGE  & 
VAN  HEEMST 

WITHERS  & 
LEES 

HABER'S  RULE 

Lethal  Toxic  Load  (LTLso) 

2.582E+10 

9.059E+09 

9.861  E+09 

9.662E+06 

9.500E+03 

Toxic  Load  Exponent  (n) 

2.75 

2.75 

2.75 

2.00 

1.00 

Probit  Slope  with  respect  to 
Toxic  Load  PS  (TL) 

2.18 

1.89 

1.15 

1.68 

6.00 

Probit  Slope  with  respect  to 
Concentration  PS  (CT) 

6.00 

5.19 

3.17 

3.35 

6.00 

n 

Distance  (kilometers) 

Prob  (%) 

Prob  (%) 

Prob  (%t 

Prob  (%1 

Prob  (%) 

0.2 

5.7 

30.5 

36.2 

71.3 

99.4 

0.5 

0.00 

0.00 

0.02 

0.31 

0.07 

1.0 

0.00 

0.00 

0.00 

0.00 

0.00 

1.5 

0.00 

0.00 

0.00 

0.00 

0.00 

2.0 

0.00 

0.00 

0.00 

0.00 

0.00 

i  n 

Distance  (kilometers) 

Normits 

Normits 

Normits 

Normits 

Normits 

0.2 

-1.5806 

-0.5096 

-0.3524 

0.5616 

2.5312 

0.5 

-7.5900 

-5.7152 

-3.5199 

-2.7349 

-3.2057 

1.0 

-12.2020 

-9.7103 

-5.9508 

-5.2169 

-7.3497 

1.5 

-14.2462 

-11.4811 

-7.0282 

-6.3283 

-9.2310 

2.0 

-15.7481 

-12.7821 

-7.8198 

-7.1263 

-10.5361 

13.2  ALOHA  Model  Runs 

Figure  1-14  through  Figure  1-16  are  examples  of 
the  ALOHA  output.  ALOHA  does  have  the 
convenience  of  producing  quick  graphic  results, 
including  time  history  plots.  But,  numeric  time 
history  output  is  not  available  from  ALOHA. 
Results  also  tend  to  be  more  conservative, 
especially  when  plotting  to  the  AEGL 
concentration  levels.  The  following  plots  used  the 
general  population  chlorine  toxicity  estimates  from 
this  report  at  the  2-min  exposure  interval  (see 
Table  7-6  of  the  main  body).  The  ALOHA  model 
does  seem  to  predict  more  pronounced  heavy-gas 
effects  compared  to  the  HPAC  results.  ALOHA 
also  has  an  embedded  1-h  or  10-km  cut-off  in  its 
calculations,  which  accounts  for  the  flat  cut-off 
features  in  the  contour  plots  shown.  The  rationale 
for  this  cut-off  is  that  meteorological  uncertainties 
cannot  provide  a  reliable  prediction  after  1  h  or 
beyond  10  km  after  the  event  has  occurred. 


14.0  Summary 

A  very  brief  and  limited  set  of  simulations  were 
conducted  for  use  as  input  to  a  comparison  of 
various  chlorine  lethality  estimates  (see  Table  I— 1 , 
page  1-7).  Results  can  vary  greatly  based  on  the 
ATD  model  used  (and  the  methodology  assump¬ 
tions  assumed  within  the  model),  input  parameters 
assumed  and  the  definition  of  toxicological  effects 
used  to  determine  population  effects,  casualties 
and/or  fatalities.  Another  important  observation  is 
that  the  graphical  output  from  neither  HPAC  or 
ALOHA  can  adequately  display  toxicity  estimates 
that  are  based  on  a  toxic  load  model  for  time- 
dependence.  When  the  toxic  load  is  properly  cal¬ 
culated  from  numerical  HPAC  output,  it  was  found 
that  some  lethality  relationships  do  produce 
probability  of  lethality,  downwind  distance  profiles 
that  are  consistent  with  the  historical  record  for 
accidental  chlorine  releases.  It  is  recommended 
that  the  developers  of  HPAC  seriously  consider  the 
addition  of  a  toxic  load  contour  option  to  HPAC 
graphical  output. 
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Figure  1-14.  ALOHA  predictions  for  low  wind  (1  m/s),  clear  sky,  nighttime  (2  a.m.),  “stable” 
condition  (Pasquill  Category  F) 
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Figure  1-15.  ALOHA  predictions  for  moderate  wind  (5  m/s),  cloudy  day,  “neutral”  condition 
(Pasquill  Category  D) 
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Figure  1-16.  ALOHA  predictions  for  low  wind  (2  m/s),  clear  sky,  daytime  (2  p.m.),  “unstable” 
condition  (Pasquill  Category  B) 
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APPENDIX  J:  STATISTICAL  ANALYSIS  OF  MICE  CHLORINE  LETHALITY  DATA  OF 
BITRON  AND  AHARONSON  (1978)— Douglas  R.  Sommerville 


J1.0  Introduction 

Mouse  chlorine  lethality  data  from  Bitron  and 
Aharonson1  were  subjected  to  a  statistical  analysis. 
Bitron  and  Aharonson  exposed  476  mice  (in 
groups  of  14)  about  evenly  divided  between  either 
of  two  concentrations  (C)  of  chlorine  vapor 
(170  ppm  or  290  ppm)  and  for  varying  periods  of 
time  (7)  (from  14  min  to  160  min  for  170  ppm  and 
from  6  min  to  3 1  min  for  290  ppm).  Their  data  was 
reevaluated  in  this  analysis  using  modem  statistical 
methods  in  order  to  better  model  how  the  median 
lethal  dosage  (LCT50)  varied  as  a  function  of 
exposure  duration.  Originally,  Bitron  and 
Aharonson  did  not  report  a  mathematical 
expression  for  the  time-dependence  of  the  toxicity. 
However,  subsequent  researchers2'4  have  fitted  a 
toxic  load  expression  (L (C"T)50  =  a  constant)5,6 
using  the  two  LT50  values  that  were  reported 
(11  min.  for  290  ppm  and  55  min  for  170  ppm).  A 
toxic  load  exponent  («)  value  of  3.5  has  been 
reported  by  ten  Berge  and  van  Heemst2  and  a  value 
of  3  from  the  methoda  used  by  Withers  and  Lees.3 
These  estimates  are  both  based  upon  an  incomplete 
picture  of  the  data,  ten  Berge  and  van  Heemst  used 
a  multifactor  probit  fit  on  a  select  portion  of  the 
original  data,  and  Withers  and  Lees  depended  on 
the  reported  median  lethal  dosages  instead  of 
analyzing  the  original  binary  response  data.  To 
address  this  deficiency,  the  complete  response 
dataset  of  Bitron  and  Aharonson  was  subjected  to  a 
multifactor  probit  analysis8  using  the  binary 
logistic  regression  routine  in  Minitab®9  From  this 
fit,  the  values  of  the  toxic  load  exponent  and  other 
important  parameters  were  estimated,  along  with 
95%  confidence  intervals. 

J2.0  Data  Collection  and  Reduction 

The  individual  percent  response  values  are  not 
listed  separately  in  the  original  report.  Instead, 
these  values  must  be  extracted  from  the  graphical 
representations  shown  in  the  Bitron  and  Aharonson 
report,  Figures  9  and  11.  In  their  Figure  9,  the 
overall  mortalities  for  individual  grouping  of  three 
to  four  exposure  durations  each  were  plotted  as  a 
function  of  days  post-exposure.  In  their  Figure  1 1 , 
the  percent  response  is  plotted  versus  the  exposure 


millimeter  ruler  was  then  used  to  determine  the 
coordinates  of  each  symbol  relative  to  the  origin  of 
the  plot.  Their  coordinates  were  then  converted 
into  values  for  numbers  of  deaths  per  exposure 
group  for  some  value  of  C  and  T  using  the 
appropriate  map  scale  for  the  enlarged  figure.  For 
the  response  data  at  290  ppm,  there  are  five 
experiments  of  100%  mortality  that  were  omitted 
from  their  Figure  11.  So,  the  response  data  was 
reconstructed  in  the  best  manner  possible.  As  a 
check  of  the  reconstructed  data,  the  total  percent 
mortality  in  their  Figure  9  for  each  grouping  of 
durations  was  consulted  and  also  subjected  to  a 
regression  analysis.  The  collected  data  are  shown 
in  Table  J-l  and  Table  J-2.  A  summary  of  percent 
mortality  values  per  exposure  duration  group  from 
their  Figure  9  is  listed  in  Table  J-3. 


aOriginally,  Withers  and  Lees  calculated  an  average  toxic  load  exponent  value  using  median  toxic  load  values  from  Bitron  and 
Aharonson  and  Weedon  et  al.7  If  only  the  data  of  Bitron  and  Aharonson  is  subjected  to  the  Withers  and  Lees  calculation 
algorithm,  a  toxic  load  exponent  value  of  3  is  obtained. 
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Table  J-1.  Mouse  chlorine  inhalation  lethality  data  from  Figure  11  of  Bitron  and  Aharonson  for 
170  ppm  of  chlorine  vapor 


c 

(ppm) 

T 

(min) 

DEATHS 

NUMBER  OF 
MICE  EXPOSED 

PERCENT 

LETHALITY 

DURATION 
GROUP  IN 
FIGURE  9 
(min) 

170 

13 

0 

14 

0.0 

22 

170 

16 

4 

14 

28.6 

22 

170 

18 

1 

14 

7.1 

22 

170 

29 

1 

14 

7.1 

22 

170 

39 

4 

14 

28.6 

52 

170 

41 

3 

14 

21.4 

52 

170 

42 

9 

14 

64.3 

52 

170 

49 

10 

14 

71.4 

52 

170 

57 

13 

14 

92.9 

52 

170 

57 

4 

14 

28.6 

52 

170 

57 

3 

14 

21.4 

52 

170 

81 

10 

14 

71.4 

120 

170 

100 

10 

14 

71.4 

120 

170 

112 

13 

14 

92.9 

120 

170 

121 

12 

14 

85.7 

120 

170 

163 

8 

14 

57.1 

120 

J-2 


Table  J-2.  Mouse  chlorine  inhalation  lethality  data  from  Figure  11  of  Bitron  and  Aharonson  for 
290  ppm  of  chlorine  vapor 


c 

(ppm) 

T 

(min) 

DEATHS 

NUMBER  OF 
MICE  EXPOSED 

PERCENT 

LETHALITY 

DURATION 
GROUP  IN 
FIGURE  9 

(min) 

290 

6 

0 

14 

0.0 

6 

290 

6 

0 

14 

0.0 

6 

290 

7.5 

4 

14 

28.6 

9 

290 

8.5 

7 

14 

50.0 

9 

290 

9.25 

7 

14 

50.0 

9 

290 

10.25 

4 

14 

28.6 

9 

290 

13 

7 

14 

50.0 

15 

290 

13 

11 

14 

78.6 

15 

290 

13 

11 

14 

78.6 

15 

290 

13 

14 

14 

100.0 

15 

290 

14 

10 

14 

71.4 

15 

290 

16.25 

14 

14 

100.0 

15 

290 

20.5 

12 

14 

85.7 

25 

290 

23.5 

12 

14 

85.7 

25 

290 

19 

14 

14 

100.0 

25 

290 

31 

14 

14 

100.0 

25 

290 

25 

14 

14 

100.0 

25 

290 

25 

14 

14 

100.0 

25 

Table  J-3.  Summary  of  mouse  chlorine  inhalation  mortality  from  Figure  9  of  Bitron  and  Aharonson 


C 

(ppm) 

DURATION 
GROUP  (min) 

DEATHS 

NUMBER  OF 
MICE  EXPOSED 

NUMBER  OF 
EXPERIMENTS 

PERCENT 

MORTALITY 

170 

22  +  8 

6 

56 

4 

10.7 

170 

52  +  13 

46 

98 

7 

46.9 

170 

120  +  40 

53 

70 

5 

75.7 

290 

6 

0 

28 

2 

0.0 

290 

9+1 

22 

56 

4 

39.3 

290 

15  +  2 

67 

84 

6 

79.8 

290 

25  +  6 

80 

84 

6 

95.2 
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J3.0  Statistical  Analysis  of  Binary 
Response  Data 

The  following  are  the  Minitab  printouts  for  the 
multifactor  probit  analysis  of  the  mouse  chlorine 
inhalation  lethality  data.  Examples  are  presented 
on  how  various  final  parameter  values  (median 
effective  dosages,  dosage  ratio  of  severe  and  lethal 
effects,  etc.)  are  calculated  from  these  printouts. 

J3.1  Nomenclature 


c 

Concentration  of  chlorine  vapor  in 
ppm  (1  mg/m3  =  2.898  x  1  ppm) 
(used  as  a  covariate) 

Cone 

Concentration  treated  as  a  factor 

Deaths 

Number  of  deaths 

logC 

Log  base  1 0  of  vapor  concentration 

logCT 

Log  base  10  of  vapor  concentration 
multiplied  by  exposure  duration 

logT 

Log  base  10  of  exposure  duration 

n 

Toxic  load  exponent 

Number 

Number  of  mice  exposed  to  indicated 
row  values  of  C  and  T 

SE 

Standard  error  of  coefficient 

T 

Exposure  duration  (in  minutes) 

Time 

Duration  treated  as  a  factor 

Z 

Normit  (Z  =  0  for  50%  response,  -1 
for  16%  response  and  1  for  84% 
response) 

J3.2  Toxic  Load  Exponent  Estimation 

To  estimate  the  toxic  load  exponent,  a  multifactor 
probit  analysis  was  performed  on  logC  and  logT 
for  two  sets  of  data:  (1)  data  from  Table  J-l  and 
Table  J-2;  and  (2)  data  from  Table  J-3.  The  ratio 
of  the  fitted  coefficients  for  logC  and  logT  (or 
kc/kt)  equals  the  toxic  load  exponent.6 
J3.2.1  Binary  Response  from  Table  J-1 
and  Table  J-2  Versus  LogC  and  LogT 

Link  Function:  Normit 
Response  Information 


Variable 

Value 

Count 

Deaths 

Event 

274 

Non-event 

202 

Number 

Total 

476 

Logistic  Regression  Table 


Predictor 

Coef 

SE  Coef 

Z 

P 

Constant 

-29.2194 

2.75200 

-10.62 

0.000 

logC 

10.6318 

1.01580 

10.47 

0.000 

logT 

3.16473 

0.303112 

10.44 

0.000 

Log-Likelihood  =  -240.542 

Test  that  all  slopes  are  zero:  G  =  167.860,  DF  =  2, 

P- Value  =  0.000 

Goodness-of-Fit  Tests 


Method 

Chi-Square  DF 

P 

Pearson 

89.6501 

24 

0.000 

Deviance 

81.4549 

24 

0.000 

Hosmer- 

18.3257 

7 

0.011 

Lemeshow 

Variance-Covariance  Matrix  for  Fit 

Constant 

logC 

logl 

7.57348 

-2.78459 

-0.729094 

-2.78459 

1.03185 

0.255530 

-0.72909 

0.25553 

0.091877 

The  LT50  values  from  this  fit  for  concentrations  of 
170  ppm  and  290  ppm  are  55  min  and  9.1  min, 
respectively.  Bitron  and  Aharonson  originally 
reported  55  min  and  1 1  min  using  the  Litchfield 
and  Wilcoxon  method11  to  analyze  their  data.  The 
probit  slope  ( kc )  for  this  fit  equals  10.6  with  95% 
fiducial  limits  of  8.64  to  12.62. 

There  is  statistically  significant  lack  of  fit,  based 
upon  three  statistical  tests  (Pearson,  Deviance  and 
Hosmer-Lemeshow).  A  review  of  the  standardized 
Pearson  residuals  identified  two  outliers:  (1)  16 
min  at  170  ppm  (4  deaths  out  of  14);  and  (2)  163 
min  at  170  ppm  (8  deaths  out  of  14).  A  second 
regression  without  the  two  outliers  produces  the 
following  fit. 

Link  Function:  Normit 
Response  Information 


Variable 

Value 

Count 

Deaths 

Event 

262 

Non-event 

186 

Number 

Total 

448 

*  NOTE  *  32  cases  were  used 

*  NOTE  *  2  cases  contained  missing  value 
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Logistic  Regression  Table 


Predictor 

Coef 

SE  Coef 

Z 

P 

Constant 

-38.5843 

3.54588 

-10.88 

0.000 

logC 

13.9054 

1.29500 

10.74 

0.000 

logT 

4.37248 

0.399308 

10.95 

0.000 

Log-Likelihood  =  -205.765 

Test  that  all  slopes  are  zero:  G  =  196.574,  DF  =  2, 
P- Value  =  0.000 

Goodness-of-Fit  Tests 


Method 

Chi-Square 

DF 

P 

Pearson 

41.3492 

22 

0.007 

Deviance 

47.7748 

22 

0.001 

Hosmer- 

5.5522 

5 

0.352 

Lemeshow 

Variance-Covariance  Matrix  for  Fit 


Constant 

logC 

logl 

12.5733 

-4.57879 

-1.29129 

-4.5788 

1.67702 

0.45500 

-1.2913 

0.45500 

0.15945 

For  the  second  fit,  at  least  one  test  (Hosmer- 
Lemeshow)  returned  a  p-value  greater  than  0.05 
signifying  that  there  is  no  lack  of  fit.  The  LT50 
values  from  this  fit  for  concentrations  of  170  ppm 
and  290  ppm  are  54  min  and  9.9  min,  respectively. 
This  is  not  drastically  different  from  the  regression 
fit  that  included  the  outliers. 

The  probit  slope  (kr)  for  this  fit  equals  13.9  with 
95%  fiducial  limits  of  11.4  to  16.4.  This  is  higher 
than  for  the  fit  that  included  the  outliers.  This  is 
not  surprising,  since  the  probit  slope  is  a  measure 
of  variance  and  the  remove  of  outliers  will  usually 
reduce  the  variance  in  the  data. 

The  present  of  statistically  significant  curvature 
was  tested  for  by  introducing  a  (logT)2  term  into 
the  model.  This  term  was  found  to  be  statistically 
significant  for  the  dataset  in  Table  J-l  and  Table 
J-3,  with  and  without  the  previously  identified 
outliers.  Curvature  in  the  logC  and  logT  fit  is  not 
unheard  of,  but  the  result  is  difficult  to  use  in 
atmospheric  transport  and  dispersion  models.12 

J3.2.2  Binary  Response  from  Table  J-3 
Versus  LogC  and  LogT 

Link  Function:  Normit 


Response  Information 


Variable 

Value 

Count 

Deaths 

Event 

274 

Non-event 

202 

Number 

Total 

476 

Logistic  Regression  Table 

Predictor 

Coef 

SE  Coef 

Z 

P 

Constant 

-32.2629 

2.83110 

-11.40 

0.000 

logC 

11.6873 

1.04248 

11.21 

0.000 

logT 

3.49093 

0.309412 

11.28 

0.000 

Log-Likelihood  =  -234.041 

Test  that  all  slopes  are  zero:  G  =  180.861,  DF  =  2, 

P- Value  =  0.000 

Goodness-of-Fit  Tests 


Method 

Chi-Square 

DF 

P 

Pearson 

19.6929 

4 

0.001 

Deviance 

25.0251 

4 

0.000 

Hosmer- 

10.7637 

4 

0.029 

Lemeshow 

Variance-Covariance  Matrix  for  Fit 

Constant 

logC 

logl 

8.01514 

-2.93985 

-0.769620 

-2.93985 

1.08676 

0.269349 

0.26935 

-0.76962 

0.095736 

The  LT50  values  from  this  fit  for  concentrations  of 
170  ppm  and  290  ppm  are  59  min  and  9.9  min, 
respectively.  Bitron  and  Aharonson  originally 
reported  55  min  and  1 1  min  using  the  Litchfield 
and  Wilcoxon  method11  to  analyze  their  data. 

There  is  statistically  significant  lack  of  fit,  based 
upon  three  statistical  tests  (Pearson,  Deviance  and 
Hosmer-Lemeshow).  A  review  of  the  standardized 
Pearson  residuals  identified  two  outliers: 

(1)  120  min  at  170  ppm  (53  deaths  out  of  70);  and 

(2)  6  min  at  290  ppm  (zero  deaths  out  of  28). 
Omitting  either  of  the  outliers  will  remove  the  lack 
of  fit.  However,  there  is  a  big  difference  between 
the  resulting  toxic  load  exponent  values  depending 
on  which  datum  is  chosen  for  omission  (2.95 
versus  3.59). 

The  presence  of  statistically  significant  curvature 
was  tested  for  by  introducing  a  (logT)2  term  into 
the  model.  This  term  was  found  to  be  statistically 
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significant,  a  result  identical  to  that  of 
Section  J3.2.1. 

J3.2.3  Calculation  of  the  Toxic  Load 
Exponent 

The  following  is  a  step  by  step  example  of 
calculating  the  toxic  load  exponent  and  associated 
confidence  limits. 

(1)  Calculation  of  toxic  load  exponent  for  first 
dataset  (Table  J-l  and  Table  J-2): 


n  =  kc  =  (10.6318)  =3.36 

kt  (3.16473) 


(J1) 


(2)  The  standard  error  of  a  ratio  needs  to  be 
calculated.  From  Mood  et  al.  (1974),  the  following 
is  given:10 


var  (al  b)= 


var(a)  var(fc) 


(2)cov(a,£>) 

ab 


StndError  =  *Jvar  (alb) 


(J2) 


(3)  Using  values  from  the  variance-covariance 
matrix,  the  variance  of  the  numerator,  var(num)  or 
var (kc),  equals  1.03185.  For  k„  the  variance  equals 
0.091877. 

(4)  Using  values  from  the  variance-covariance 
matrix,  the  covariance  of  the  numerator  and  the 
denominator,  co \{kc,  kt),  equals  0.255530. 

(5)  Thus,  the  standard  error  (using  eq  A3)  equals: 


var (alb). 


(10.6318)2 

(1.03185)  (0.091877) 

(2)(0.255530) 

(3.16473)2 

(10.6318)2  (3.16473)2 

( 10.6318)(  3.16473) 

(J3) 


StndError  =  var(a/b)  =  0.187 


(6)  The  approximate  95%  confidence  limits  for  the 
toxic  load  exponent  now  equal: 

-(1.96)(StndErr)  <  n  <  +  (1.96)(StndErr) 

or 

3.359  -  (1.96)(0.187)  <  n  <  3.359  +  (1.96)(0.187)  (J4) 

or 

2.992  <  Ratio  <  3.727 


The  above  calculations  were  repeated  for  the  fit  of 
Section  J3.2.1  (without  two  outliers).  The  toxic 
load  exponent  was  found  to  equal  3.18  with  95% 
Cl  of  2.90  to  3.46.  For  the  fit  from  Section  J3.2.2, 
the  exponent  was  found  to  equal  3.35  with  95%  Cl 
of  3.01  to  3.68. 


J3.3  Probit  Slope  (kt)  Estimation 

To  estimate  the  probit  slope  for  exposure  duration 
(kt),  a  multifactor  probit  analysis  was  performed  on 
covariate  logT  nested  with  the  factor  Cone  for  two 
sets  of  data:  (1)  data  from  Table  J-l  and  Table  J-2; 
and  (2)  data  from  Table  J-3.  The  estimates  for  kt 
are  shown  in  Table  J-4,  along  with  the  originally 
reported  values  from  Bitron  and  Aharonson.  The 
reverse  analysis  (covariate  logC  nested  with  the 
factor  Time)  could  not  be  performed,  due  to  having 
only  two  unique  values  for  concentration  (170  ppm 
and  290  ppm).  Thus,  for  a  kc  estimate,  one  of  the 
approaches  shown  in  Section  J3.2  must  be  used. 

For  the  fit  of  logT  nested  within  C,  there  was  no 
lack  of  fit  for  any  of  the  datasets  analyzed.  Also, 
for  both  datasets,  the  probit  slope  values  are 
different  from  each  other  with  statistical 
significance.  This  is  consistent  with  the  significant 
curvature  found  in  the  logC  and  logT  fit  of  Section 
J3.2.  There  is  agreement  between  the  probit  slope 
estimates  from  this  appendix  and  what  was 
originally  reported  by  Bitron  and  Aharonson. 

J4.0  Discussion  and  Summary 

The  presence  of  significant  curvature  in  the  logC 
and  logT  fit  (see  Section  J3.2)  is  problematic. 
Flowever,  the  curvature  could  be  an  artifact  of  an 
improperly  designed  test  matrix.  The  probit  slope 
is  a  measure  of  all  variance  present  in  a  study,  not 
just  of  the  variability  in  individual  toxic  responses. 
For  example,  if  all  the  runs  for  each  concentration 
were  performed  together  (i.e.,  all  the  170-ppm 
experiments  were  performed  before  any  of  the 
290-ppm  experiments),  then  there  could  be  a  batch 

Table  J-4.  Summary  of  probit  slope 
(exposure  duration)  estimates  from  mouse 
chlorine  inhalation  mortality  of  Bitron  and 
Aharonson 


DATASET 

C 

(ppm) 

PROBIT 
SLOPE  {ki) 
(BASE  10) 

95% 

FIDUCIAL 

LIMITS 

Tables 

J1&J2 

170 

2.20 

1.55  to  2.84 

290 

5.67 

4.40  to  6.94 

Table  J3 

170 

2.56 

1.85  to  3.27 

290 

5.25 

4.12  to  6.37 

Bitron  and 
Aharonson 

170 

2.5 

NR 

290 

4.7 

NR 
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effect  due  to  changes  in  laboratory  procedures  (i.e., 
different  animal  handlers,  different  batches  of 
chlorine,  seasonal  effects,  etc.).  Also,  only  two 
concentration  values  were  investigated,  preventing 
the  testing  for  curvature  as  a  function  of 
concentration.  So,  for  the  moment,  a  simplified 
approach  (assuming  linearity  in  the  logC  and  logT 
fit)  is  warranted  until  such  curvature  is  reproduced 
in  a  future  study. 

Using  three  different  datasets  (Table  J-l  and  Table 
J-2;  Table  J-l  and  Table  J-2  with  two  outliers 
removed,  and  Table  J — 3),  remarkably  similar 
values  for  the  toxic  load  exponent  were  obtained. 
For  a  final  estimate  for  both  the  toxic  load 
exponent  and  the  probit  slope  ( kc ),  the  fit  (with  the 


two  outliers)  in  Section  J3.2.1  should  be  used  for 
two  reasons: 

(1)  The  original  notes  from  Bitron  and  Aharonson 
are  not  available,  thus  identification  of  suspect 
experiments  would  be  difficult.  Omitting  one 
outlier  over  another  can  greatly  influence  the 
final  estimate  for  the  toxic  load  exponent. 

(2)  The  exposure  duration  values  in  Table  J-3  are 
averages  of  the  actual  durations  that  were  used, 
which  would  artificially  reduce  the  lethal 
response  variance. 

From  the  logC  and  logT  fit  (Section  J3.2.1),  the 
toxic  load  exponent  equals  3.36  with  95% 
confidence  interval  of  2.99  to  3.73.  The  probit 
slope  (kc)  equals  10.6  with  95%  fiducial  limits  of 
8.64  to  12.62. 
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